Automated sample weight measurement via optical inspection

ABSTRACT

A method includes the steps collecting measurement data of a sample utilizing an adaptable inspection unit or while the sample is in-flight, determining a volume or area of the sample based at least in part on the measurement data, and calculating a weight of the sample based at least in part on the volume or area of the sample. The measurement data includes a captured image that includes a plurality of pixels. The determining of the volume of the sample includes determining the number of pixels in the captured image that display a portion of the sample, or determining the maximum number of consecutive pixels that display a portion of the sample in two or three dimensions.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority under 35U.S.C. 120 from nonprovisional U.S. patent application Ser. No.18/118,319, entitled “AUTOMATED SAMPLE WEIGHT MEASUREMENT VIA OPTICALINSPECTION”, filed on Mar. 7, 2023, the subject matter of which isincorporated by reference. application Ser. No. 18/118,319 is in turn acontinuation-in-part and claims priority under 35 U.S.C. 120 fromnonprovisional U.S. patent application Ser. No. 18/106,211, entitled“AUTOMATED INSPECTION DATA COLLECTION FOR MACHINE LEARNINGAPPLICATIONS”, filed on Feb. 6, 2023, the subject matter of which isincorporated by reference. application Ser. No. 18/106,211 is in turn acontinuation-in-part and claims priority under 35 U.S.C. 120 fromnonprovisional U.S. patent application Ser. No. 17/985,747, entitled“ADAPTABLE INSPECTION AND SORTING UNIT”, filed on Nov. 11, 2022, thesubject matter of which is incorporated by reference. application Ser.No. 17/985,747 is in turn a continuation-in-part and claims priorityunder 35 U.S.C. 120 from nonprovisional U.S. patent application Ser. No.17/979,618, entitled “PRODUCT TARGET QUALITY CONTROL SYSTEM WITHINTELLIGENT SORTING”, filed on Nov. 2, 2022, the subject matter of whichis incorporated by reference. application Ser. No. 17/979,618 is in turna continuation-in-part and claims priority under 35 U.S.C. 120 fromnonprovisional U.S. patent application Ser. No. 17/967,621, entitled“PRODUCT TARGET QUALITY CONTROL SYSTEM”, filed on Oct. 17, 2022, thesubject matter of which is incorporated by reference. application Ser.No. 17/967,621 is in turn a continuation-in-part and claims priorityunder 35 U.S.C. 120 from nonprovisional U.S. patent application Ser. No.17/735,263, entitled “SUB STREAM AUTO SAMPLING”, filed on May 3, 2022,the subject matter of which is incorporated by reference. applicationSer. No. 17/735,263, is in turn a continuation and claims priority under35 U.S.C. 120 from nonprovisional U.S. patent application Ser. No.17/132,500, entitled “SUB STREAM AUTO SAMPLING”, filed on Dec. 23, 2020,the subject matter of which is incorporated by reference. applicationSer. No. 17/132,500, is in turn a continuation-in-part and claimspriority under 35 U.S.C. 120 from nonprovisional U.S. patent applicationSer. No. 16/861,156, entitled “VACUUM ADAPTABLE SORTER UNIT FOR EXISTINGPROCESSING LINES”, filed on Apr. 28, 2020, the subject matter of whichis incorporated by reference. application Ser. No. 16/861,156, is inturn a continuation-in-part and claims priority under 35 U.S.C. 120 fromnonprovisional U.S. patent application Ser. No. 16/257,056, entitled“ADAPTABLE SORTER UNIT FOR EXISTING PROCESSING LINES”, filed on Jan. 24,2019, the subject matter of which is incorporated by reference.application Ser. No. 16/257,056, is in turn a continuation-in-part andclaims priority under 35 U.S.C. 120 from nonprovisional U.S. patentapplication Ser. No. 16/031,956, entitled “QUALITY INSPECTION DATADISTRIBUTED LEDGER”, filed on Jul. 10, 2018, the subject matter of whichis incorporated by reference. application Ser. No. 16/031,956, is inturn a continuation-in-part and claims priority under 35 U.S.C. 120 fromnonprovisional U.S. patent application Ser. No. 15/995,126, entitled“INSPECTION DEVICE CONTROLLED PROCESSING LINE SYSTEM”, filed on Jun. 1,2018, the subject matter of which is incorporated by reference.application Ser. No. 15/995,126, in turn is a continuation and claimspriority under 35 U.S.C. § 120 from nonprovisional U.S. patentapplication Ser. No. 15/817,240, entitled “INSPECTION DEVICE CONTROLLEDPROCESSING LINE SYSTEM,” filed on Nov. 19, 2017, the subject matter ofwhich is incorporated herein by reference. application Ser. No.15/817,240, in turn, is a continuation-in-part and claims priority under35 U.S.C. 120 from nonprovisional U.S. patent application Ser. No.15/219,870, entitled “IN-FLIGHT 3D INSPECTOR”, filed on Jul. 26, 2016,the subject matter of which is incorporated by reference.

TECHNICAL FIELD

The present invention generally relates to systems and methods formeasuring sample weight via optical inspection.

BACKGROUND INFORMATION

In the field of product inspection, measuring the quality and quantityof the product is very valuable. In addition to measuring the quantityin sheer numbers of units, it would also be valuable to measure thequantity in terms of weight, both per unit and overall total weight ofall the product. For example, in the field of almond inspection, itwould be valuable to measure the maximum, minimum, average weight of agroup of almonds, and the total sum weight of all almonds in the group.This type of data coupled with the quality data can provide greatinsight into the final value of the group of inspected almonds. Thisholds true with respect to inspection of other foods and products. Amethod for automatically measuring sample weight via optical inspectionis provided below.

SUMMARY

In a first novel aspect, a method includes the steps of collectingmeasurement data of a sample utilizing an adaptable inspection unit orwhile the sample is in-flight, determining a volume of the sample basedat least in part on the measurement data, and calculating a weight ofthe sample based at least in part on the volume. The measurement dataincludes a captured image that includes a plurality of pixels. Thedetermining of the volume of the sample includes determining the numberof pixels in the captured image that display a portion of the sample ordetermining the maximum number of consecutive pixels that display aportion of the sample in two or three dimensions.

In a second novel aspect, a method includes the steps of collectingmeasurement data of a sample utilizing an adaptable inspection unit orwhile the sample is in-flight, determining an area of the sample basedat least in part on the measurement data, and calculating a weight ofthe sample based at least in part on the area. The measurement dataincludes a captured image that includes a plurality of pixels. Thedetermining of the area of the sample includes determining the number ofpixels in the captured image that display a portion of the sample ordetermining the maximum number of consecutive pixels that display aportion of the sample in two dimensions.

In one example, the measurement data includes a captured image.

In another example, the measurement data is measured by an opticalinspector.

Further details and embodiments and techniques are described in thedetailed description below. This summary does not purport to define theinvention. The invention is defined by the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, where like numerals indicate like components,illustrate embodiments of the invention.

FIG. 1 is a first diagram of the in-flight 3D inspector 1 view from afirst perspective.

FIG. 2 is a second diagram of the in-flight 3D inspector 1 view from asecond perspective.

FIG. 3 is a third diagram of the in-flight 3D inspector 1 view from aright side view.

FIG. 4 is a fourth diagram of the in-flight 3D inspector 1 view from aleft side view.

FIG. 5 is a diagram of the in-flight 3D inspector 1 illustrating thepath a sample travels through the in-flight 3D inspector 1.

FIG. 6 is a diagram of a double stereo camera system configuration withtriggering.

FIG. 7 is an image captured by a first camera of the double stereocamera system.

FIG. 8 is an image captured by a second camera of the double stereocamera system.

FIG. 9 is an image captured by a third camera of the double stereocamera system.

FIG. 10 is an image captured by a fourth camera of the double stereocamera system.

FIG. 11 is a flowchart of an in-flight 3D inspector.

FIG. 12 is a flowchart of an in-flight 3D inspector with defectprocessing.

FIG. 13 is a diagram of an inspection device.

FIG. 14 is a diagram of an inspection data communication system.

FIG. 15 is a diagram of a command communication system.

FIG. 16 is a diagram of an inspection data control system using a remotecomputing device.

FIG. 17 is a diagram of an inspection data control system of multipleprocessing lines using a remote computing device.

FIG. 18 is a diagram of a first example of inspection data.

FIG. 19 is a diagram of a second example of inspection data.

FIG. 20 is a diagram of a third example of inspection data.

FIG. 21 is a diagram of a first example of a command based on inspectiondata.

FIG. 22 is a diagram of a second example of a command based oninspection data.

FIG. 23 is a diagram of a third example of a command based on inspectiondata.

FIG. 24 is a flowchart illustrating the operation of an inspection datacommunication system.

FIG. 25 is a flowchart illustrating the operation of a commandcommunication system.

FIG. 26 is a flowchart illustrating a first example of the operation ofan inspection data communication system using a remote computing device.

FIG. 27 is a flowchart illustrating a second example of the operation ofan inspection data communication system.

FIG. 28 is a quality inspection data distributed ledger flowchart.

FIG. 29 is a diagram of a quality inspection data block in a qualityinspection distributed ledger.

FIG. 30 is a diagram of a conveyor for manual inspection or sorting.

FIG. 31 is a diagram of a conveyor with an adaptable inspection unitattached to the conveyor.

FIG. 32 is a diagram of a conveyor with an adaptable sorting unitattached to the conveyor.

FIG. 33 is a diagram of a conveyor with an adaptable inspection unitattached to the ceiling above the conveyor.

FIG. 34 is a diagram of a conveyor with an adaptable sorting unitattached to a ceiling above the conveyor.

FIG. 35 is a diagram of a conveyor with an adaptable inspection unitattached to a mounting stand.

FIG. 36 is a diagram of a conveyor with an adaptable sorting unitattached to a mounting stand.

FIG. 37 is a diagram of a conveyor with an adaptable inspection unitattached to the conveyor sidewall. The adaptable inspection unit can beattached permanently or temporarily to the conveyor sidewall.

FIG. 38 is a diagram of a conveyor with an adaptable inspection unitattached to the conveyor and an adaptable sorting unit attached to theconveyor.

FIG. 39 is a block diagram of an adaptable inspection unit.

FIG. 40 is a block diagram of an adaptable sorter unit.

FIG. 41 is a flowchart illustrating the operations performed by anadaptable inspection unit.

FIG. 42 is a flowchart illustrating the operations performed by anadaptable sorting unit.

FIG. 43 is a diagram of an adaptable inspection unit and vacuumadaptable sorter unit that utilizes pressurized air.

FIG. 44 is a side-view diagram of a vacuum adaptable sorter unit thatutilizes pressurized air.

FIG. 45 is a diagram illustrating the operation of a Venturi vacuum. A

FIG. 46 is a front-view diagram of an adaptable inspection unit andvacuum adaptable sorter unit utilizing pressurized air with x-y-zlocation adjustment.

FIG. 47 is a top-down diagram of an adaptable inspection unit and vacuumadaptable sorter unit utilizing pressurized air with x-y-z locationadjustment.

FIG. 48 is a front-view diagram of an adaptable inspection unit andarray of fixed location vacuum adaptable sorter units utilizingpressurized air.

FIG. 49 is a top-down diagram of an adaptable inspection unit and arrayof fixed location vacuum adaptable sorter units utilizing pressurizedair.

FIG. 50 is a perspective diagram of an adaptable inspection unit andarray of fixed location vacuum adaptable sorter units utilizingpressurized air.

FIG. 51 is a flowchart describing the steps of enabling a vacuumadaptable sorter unit that utilizes pressurized air.

FIG. 52 is a perspective diagram of sub stream inspection system.

FIG. 53 is a diagram of a sub stream inspection system.

FIG. 54 is a flowchart of a sub stream inspection system.

FIG. 55 is a diagram of a sub stream inspection and weighing system.

FIG. 56 is a flowchart of a sub stream inspection and weighing system.

FIG. 57 is a diagram of a sub stream weighing and inspection system.

FIG. 58 is a flowchart of a sub stream weighing and inspection system.

FIG. 59 is a diagram of a sub stream inspection and collection system.

FIG. 60 is a flowchart of a sub stream inspection and collection system.

FIG. 61 is a diagram of a sub stream inspection, weighing and collectionsystem.

FIG. 62 is a flowchart of a sub stream inspection, weighing andcollection system.

FIG. 63 is a diagram of a sub stream weighing, inspection, andcollection system.

FIG. 64 is a flowchart of a sub stream weighing, inspection, andcollection system.

FIG. 65 is a flowchart diagram of a target quality control system withintelligent source control.

FIG. 66 is an operational diagram of a first target quality controlsystem.

FIG. 67 is an operational diagram of a second target quality controlsystem.

FIG. 68 is an operational diagram of a third target quality controlsystem.

FIG. 69 is a flowchart of a target quality control system.

FIG. 70 is a flowchart of a target quality control system.

FIG. 71 is a flowchart diagram of a target quality control system withintelligent inspection and sorting.

FIG. 72 is an operation diagram of a first target quality control systemusing intelligent inspection and sorting.

FIG. 73 is an operation diagram of a second target quality controlsystem using intelligent inspection and sorting.

FIG. 74 is a perspective view of an inspection and sorting productionline.

FIG. 75 is an operation diagram of a third target quality control systemusing intelligent inspection and sorting.

FIG. 76 is an operation diagram of a fourth target quality controlsystem using intelligent inspection and sorting.

FIG. 77 is an operation diagram of a fifth target quality control systemusing intelligent inspection and sorting.

FIG. 78 is a flowchart of a target quality control system usingintelligent inspection and sorting.

FIG. 79 is a flowchart of a target quality control system usingintelligent inspection and sorting.

FIG. 80 is a flowchart of a target quality control system usingintelligent inspection and sorting.

FIG. 81 is an operation diagram of a first target quality control systemusing intelligent inspection and sorting as well as output productinspection.

FIG. 82 is an operation diagram of a second target quality controlsystem using intelligent inspection and sorting as well as outputproduct inspection.

FIG. 83 is a flowchart of a target quality control system usingintelligent inspection and sorting as well as output product inspection.

FIG. 84 is a diagram of a conveyor with an adaptable inspection andsorting unit attached to the conveyor.

FIG. 85 is a diagram of a conveyor with an adaptable inspection andsorting unit attached to the ceiling above the conveyor.

FIG. 86 is a diagram of a conveyor with an adaptable inspection andsorting unit attached to the floor below the conveyor.

FIG. 87 illustrates another bracket geometry that can be utilized tomount the adaptable inspection and sorting unit to a conveyor.

FIG. 88 is a diagram of a conveyor with an adaptable inspection andsorting unit attached to a mounting stand.

FIG. 89 is a diagram of a conveyor with an adaptable inspection andsorting unit attached to the conveyor sidewall.

FIG. 90 illustrates a block diagram of an adaptable inspection unit thatincludes an attachment mechanism, an inspection sensor device, a dataport, a sorting device, and a power port.

FIG. 91 is a flowchart illustrating the operations performed by anadaptable inspection and sorting unit.

FIG. 92 is a flowchart illustrating a first method of automatedinspection data collection for machine learning applications.

FIG. 93 is a flowchart illustrating a second method of automatedinspection data collection for machine learning applications.

FIG. 94 is a flowchart illustrating a first method of determining aconfidence value threshold for automated inspection data collection formachine learning applications.

FIG. 95 is a flowchart illustrating a second method of determining aconfidence value threshold for automated inspection data collection formachine learning applications.

FIG. 96 is a system diagram of a first system configured to performautomated inspection data collection for machine learning applications.

FIG. 97 is a system diagram of a second system configured to performautomated inspection data collection for machine learning applications

FIG. 98 is a diagram illustrating sample size measurement along a firstplane by pixel counting.

FIG. 99 is a diagram illustrating sample size measurement along a firstplane by pixel bounding box calculation.

FIG. 100 is a diagram illustrating sample size measurement along asecond plane by pixel counting.

FIG. 101 is a diagram illustrating sample area measurement along asecond plane by pixel bounding box calculation.

FIG. 102 is a flowchart illustrating the steps of automated sample areameasurement.

FIG. 103 is a flowchart illustrating the steps of automated sample areameasurement.

FIG. 104 is a flowchart illustrating the steps of automated sample areameasurement.

FIG. 105 is a flowchart illustrating the steps of automated samplevolume measurement.

FIG. 106 is a flowchart illustrating the steps of automated sampleweight measurement.

DETAILED DESCRIPTION

Reference will now be made in detail to background examples and someembodiments of the invention, examples of which are illustrated in theaccompanying drawings. In the description and claims below, relationalterms such as “top”, “down”, “upper”, “lower”, “top”, “bottom”, “left”and “right” may be used to describe relative orientations betweendifferent parts of a structure being described, and it is to beunderstood that the overall structure being described can actually beoriented in any way in three-dimensional space.

Due to the drawbacks of human visual inspection, an automated inspectoris needed to quickly, inexpensively, and accurately detect defectspresent in objects such as tree nuts, tablets, screws and many othertypes of objects. Some of the most important features of such anautomatic inspector include: cost, number of objects inspected perminute, accuracy of defect detection, reliability of defect detectionand ease of use with minimal user training.

FIG. 1 is a first diagram of the in-flight 3D inspector 1 view from afirst perspective. The in-flight 3D inspector 1 includes a display 2, adisplay support arm 3, a sample input funnel 4, a power switch 5, anoptical system mounting frame 6, an axial fan 7, a first light source 9,a second light source 8, an RJ-45 connector 10, a collector bin 11, anda computer system 12. The display 2 outputs information from thecomputer system 12 to a human user looking at the display. The displaysupport arm 3 attaches the display 2 to the in-flight 3D inspector 1. Inone example, the display support arm is adjustable with two hinges asshown in FIG. 1 . In another example, the display support arm 3 isadjustable in additional dimensions (not shown in FIG. 1 ). The sampleinput funnel 4 is where samples are input to the in-flight 3D inspector.Power switch 5 is used by a human user to turn on (or off) the in-flight3D inspector. The light sources are mounted to the optical systemmounting frame 6. The axial fan 7 is used to create positive pressure ina camera enclosure (not shown in FIG. 1 ). In one example, the axial fan7 is coupled to a first hose that directs air flow to a first cameraenclosure and is coupled to a second hose that directs air flow to asecond camera enclosure (not shown). The hoses can be fixed or flexiblehoses made of various materials including various plastics, fiberglassand metal materials. In this fashion, positive pressure in each cameraenclosure is created. The positive pressure prevents debris fromentering the camera enclosures and settling on any of the cameras. RJ-45connector 10 is configured to receive an RJ-45 cable connected to alocal network and electrically connect the RJ-45 cable to a networkinput port included on the computer system 12. The RJ-45 cable may be anEthernet cable (not shown). Via the RJ-45 connector 10 and a RJ-45Ethernet cable, the computer system 12 can connect to a local network orthe public internet. The computer system 12 may also include a wirelessnetworking card (not shown) that allows computer system 12 to wirelesslycommunicate (i.e. WiFi or cellular connection) with a network withoutthe need for a wired connection. The collector bin 11 is configured tocollect samples that have completed their path through the in-flight 3Dinspector.

FIG. 2 is a second diagram of the in-flight 3D inspector 1 view from asecond perspective. FIG. 2 illustrates a sample chute 13 that isconfigured to guide a sample from the sample input funnel 4. Thelocation of a power management module 14 is also shown in FIG. 14 . Thepower management module 14 receives input power from the local powergrid and generates power signals for the various electrical componentsoperating within the in-flight 3D inspector 1. For example, the powermanagement module 14 generates a power signal that is used to power thevarious light sources, the various cameras (not shown), the axial fan,the display and the computer system. In one example, the powermanagement module 14 includes a battery which can be used to operate thein-flight 3D inspector when power from the local power grid is lost.

FIG. 3 is a third diagram of the in-flight 3D inspector 1 view from aright side view. FIG. 3 shows a first camera pair 18 and a second camerapair 19. FIG. 3 also illustrates that sample chute 13 is aligned withthe midpoint between the first camera pair 18 and the second camera pair19. The physical arrangement of the first camera pair 18 and the secondcamera pair 19 is illustrated in FIG. 6 . FIG. 6 illustrates that thefirst camera pair 18 includes a first camera 21 and a second camera 22.The second camera pair 19 includes a third camera 23 and a fourth camera24. All four cameras are focused on the same focal plane. The focalplane is located at the midpoint between the first camera pair 18 andthe second camera pair 19. As discussed above regarding FIG. 3 , thechute is also aligned with the midpoint between the first camera pair 18and the second camera pair 19.

Camera Positioning

The four cameras are positioned such that each camera is focused on thefocal plane. Each camera utilizes a lens to focus on the focal plane. Inone example, wide angle lenses are used by each camera. One example of awide angle lens is FL-BC1618-9M Ricoh lens. This wide angle lens has aformat size of 1″ format, a focal length of sixteen millimeters, amaximum aperture ratio of 1:1.8, an iris range of 1.8 to 16, and aresolution of nine mega-pixels. Other types of lenses may be used toachieve the necessary focus of each camera on the focal plane.

FIG. 4 is a fourth diagram of the in-flight 3D inspector 1 view from aleft side view. FIG. 4 illustrates that a third light source 15 and afourth light source 16 are also included in the in-flight 3D inspector1. In one example, the first, second, third and fourth light sources aremounted to the optical system mounting frame 6. In another example, thelight sources are mounted directly to outer frame of the in-flight 3Dinspector 1 (not shown). After reading of the present disclosure, oneskilled in the art will readily appreciate the various ways that lightsources and cameras can be physically mounted within the in-flight 3Dinspector 1.

FIG. 5 is a diagram of the in-flight 3D inspector 1 illustrating thepath a sample travels through the in-flight 3D inspector 1. First, asample 17 is placed into the sample input funnel 4. The sample inputfunnel 4 directs the sample 17 to sample chute 13. In one example, thesample input funnel 4 is configured to vibrate such that sample 17 isdirected toward sample chute 13. Sample chute 13 directs the sample 17to a focal plane where the first camera pair 18 and the second camerapair 19 are both focused. In-flight 3D inspector 1 may be used togenerate images of various types of samples, such as tree nuts, apeanuts, tablets, screws, and washers.

Triggering System

Before the sample 17 reaches the focal plane, a trigger senses thepresence of the sample 17 near the sample chute 13 and generates atrigger signal. In one example, the trigger is attached to the samplechute 13 and includes an optical transmitter and an optical receiver. Inoperation, the sample 17 interferes with the light traveling between theoptical transmitter and the optical receiver as sample 17 travels alongsample chute 13. This interference in received light is sensed by theoptical receiver when the transmitted light does not irradiate theoptical receiver. In response to detecting the interference in receivedlight, the trigger generates a trigger signal. The trigger signal can bean electric signal that propagates along a conductor, or the triggersignal can be an electro-magnetic signal that propagates across freespace to a receiving terminal. The duration between the time when thetrigger signal is generated and the time when the sample 17 intersectsthe focal plane is based on where the trigger is located relative to thefocal plane of the camera pairs. Once the trigger location is selectedthe duration between the time when the trigger signal is generated andthe time when the sample 17 intersects the focal plane can beempirically measured or calculated. Once the duration between when thetrigger signal is generated and the time when the sample 17 intersectsthe focal plane has been determined, the trigger signal can be used todetermine the future time when the sample 17 will intersect the focalplane. This timing information can be used to properly control thevarious light sources and cameras in the in-flight 3D inspector.

The trigger is not shown in FIG. 5 . However, a system diagram of thetriggering system is illustrated in FIG. 6 . FIG. 6 is a diagram of adouble stereo camera system configuration with triggering. Thetriggering system includes trigger 30, controller 31 and/or computersystem 12, cameras 21-24 and light sources 8-9 and 15-16. In oneexample, the trigger signal 32 (i) causes light sources 8, 9, 15, and 16to turn on, and (ii) causes the first camera pair 18 and the camera pair19 to capture an image when the sample 17 intersects in the focal plane.In another example, light sources 8, 9, 15 and 16 are already on and thetrigger signal 32 only causes the first camera pair 18 and the camerapair 19 to capture an image when the sample 17 intersects in the focalplane.

In a first embodiment, the trigger signal is communicated from thetrigger 30 to a controller 31 that controls when the first camera pair18 and the second camera pair 19 capture images. In a second embodiment,the trigger signal 32 is communicated from the trigger 30 directly tothe first camera pair 18 and the second camera pair 19 and causes thecamera pairs 18 and 19 to capture images. In a third embodiment, thetrigger signal 32 is communicated from the trigger 30 to computer system12 that controls when the first camera pair 18 and the second camerapair 19 capture images.

In a fourth embodiment, the trigger signal is communicated from thetrigger 30 to a controller 31 that controls when the light sources 8-9and 15-16 are turned on. The controller 31 acts as a switch thatconnects an output power terminal of a power supply included in powermanagement module 14 to a power input terminal of each light source 8-9and 15-16. The controller switch turns ON the light sources in responseto receiving the trigger signal. After the sample has passed though thefocal plane, the controller turns OFF the light sources by disconnectingthe output power terminal of the power supply from the power inputterminal of each light source.

In a fifth embodiment, the trigger signal 32 is communicated from thetrigger 30 directly to the light sources 8-9 and 15-16 and causes thelight sources 8-9 and 15-16 to turn ON. In this embodiment, each lightsource 8-9 and 15-16 is configured to receive a power signal and anON/OFF signal. The ON/OFF signal is controlled by the trigger signal.The light sources may include a timer circuit that is used to turn OFFthe light sources after the sample has passed through the focal plane.

In a sixth embodiment, the trigger signal 32 is communicated from thetrigger 30 to computer system 12 that controls when the light sources8-9 and 15-16 are turn on. In this embodiment, each light source 8-9 and15-16 is configured to receive a power signal and an ON/OFF signal. TheON/OFF signal is output by the computer system 12 in response toreceiving the trigger signal from the trigger.

The light sources may be controlled such that the light sources turn onafter the camera shutters are opened and turn off before the camerashutters are closed.

Controller 31 may be configured to communicate with computer system 12via an RS232 communication link, an Ethernet communication link, aUniversal Serial Bus (USB) communication link, or any other availabledata communication links.

When the sample 17 travels through the focal plane, sample 17 is notcontacting any surface. At this point in time, the light sources 8-9 and15-16 are turned on and the first camera pair 18 and the second camerapair 19 capture at least one image of the sample. Each camera capturesan image from a unique angle at the same moment in time as the sampletravels through the focal plane. FIG. 7 is an image captured by a firstcamera of the double stereo camera system. FIG. 8 is an image capturedby a second camera of the double stereo camera system. FIG. 9 is animage captured by a third camera of the double stereo camera system.FIG. 10 is an image captured by a fourth camera of the double stereocamera system. Each of these images is stored on a memory device locatedon the in-flight 3D inspector. On one example, the memory device islocated within the computer system 12. It is noted that the capturedimages may only be temporarily stored on a memory device within thein-flight 3D inspector before being communicated across a network toanother storage device located outside of the in-flight 3D inspector.For example, the captured images stored on a storage device within thecomputer system 12 may be communicated across RJ-45 connector 10 and alocal network to another storage device not included in the in-flight 3Dinspector. In this fashion, multiple images of the sample 17 arecaptured from four different angles at the same moment while the sample17 is traveling through the focal plane while not in contact with anysurface.

Capturing of images while the sample is not contacting any surfaceprovides a great benefit. When the sample is not contacting any surface,images of each surface of the sample can be collected at the same momentin time. This is not possible in other image capturing systems. Forexample, when a sample is moved along a conveyer belt image of only oneside of the sample may be captured at any one moment in time. View ofthe other side of the sample is blocked by the conveyer belt andtherefore cannot be captured at the same moment in time. Capturingimages of all surfaces of the sample at the same moment in time allowsfor generation of high quality 3D images of the sample. When images ofvarious surfaces of the sample are taken at different moments in time,proper alignment of images is very difficult, requires additionalprocessing and result in 3D images with lower quality due to introducederror.

The cameras communicate the captured images to the controller 31 orcomputer system 12 via bus. In one example, the bus is a UniversalSerial Bus (USB). In another example, the bus is an IEEE 1394 “FireWire”bus.

In one example, the cameras (also referred to herein as an “imagecapturing device” or “optical receiver”) are Charged Coupled Device(CCD) cameras. In another example, the cameras (also referred to hereinas an “image capturing device” or “optical receiver”) are ComplementaryMetal-Oxide Semiconductor (CMOS) cameras. In yet another example, thecameras (also referred to herein as a “image capturing device” or“optical receiver”) are Indium Gallium Arsenide (InGaAs) cameras thatare capable of measuring Short Wave Infra Red (SWIR) light.

Either line scan cameras or area scan cameras can be used to implementan in-flight 3D inspector. A line scan cameras contain a single row ofpixels used to capture data very quickly. As the object moves past thecamera, a complete image can be reconstructed in software line by line.Area scan cameras contain a matrix of pixels that capture an image of agiven scene. They are more general purpose than line scan cameras andoffer easier setup and alignment.

It is noted herein that the light sources may each include a separatepower source that drives the light when a control signal is received.Alternatively, each light source may be configured in an always on statewhere the power input terminal on each light source is coupled to anoutput terminal of a power supply where the output of the power supplyis controlled by a control signal.

It is noted that the sample chute 13 is only one example how the samplecan be directed to the focal plane. In a first alternative embodiment,the sample can be directed to the focal plane by use of a conveyer belt.In this first alternative embodiment, the sample would be directed fromthe sample input funnel to the conveyer belt, which in turn would propelthe sample off the edge of the conveyer belt toward the focal plane. Ina second alternative embodiment, the sample can be directed to the focalplane by use of an airburst. In this second alternative embodiment, thesample would be directed proximate to an airburst source, which in turnwould propel the sample toward the focal plane. One example of anairburst source is a highly pressurized air tank connected to anelectronically controlled valve, which outputs an airburst momentarilywhile the valve is open.

Sample Collection/Sorting

Once the sample 17 passes the focal plane, the sample 17 falls intocollector bin 11. In one example, a collector bucket 20 is placed incollector bin 11. In this example, the sample 17 falls into thecollector bucket 20. Additional samples placed into sample input funnel4 make their way through the in-flight 3D inspector and eventually alsofall into collector bucket 20. Once all samples have passed through thein-flight 3D inspector, a user can remove all samples by removing thecollector bucket 20 from the collector bin 11.

In another example, a collector bucket 20 is not placed in collector bin11. Rather, collector bin 11 is coupled to a sample sorting machine (notshown). In this example, the samples that pass through the in-flight 3Dinspector are routed into different bins. The bin each sample is routedinto is based on the images captured of the sample. In the event thatthe images of the sample indicate that the sample has a first type ofdefect, then the sample is routed to a first bin. In the event that theimages of the sample indicate that the sample has a second type ofdefect, then the sample is routed into a second bin. Alternatively, inthe event that the images of the sample indicate that the sample doesnot have any defects, then the sample is routed to a third bin. Thesorting machine can route the samples using various different methods. Afirst method of routing includes using a burst of air to redirect thetrajectory of a sample as it falls into the collector bin. A secondmethod of routing includes using a mechanically controlled flap toredirect the trajectory of a sample as it falls into the collector bin.

3D Image Generation

Once the images are captured from each of the cameras, a 3D image of thesample can be created. In one example, the 3D image is generated by thecomputer system 12 included in the in-flight 3D inspector. In anotherexample, the 3D image is generated by another computer system notincluded in the in-flight 3D inspector after the images are communicatedacross a network from the in-flight 3D inspector to the computer systemnot included in the in-flight 3D inspector.

The images captured by the first camera pair 18 are used to create a 3Dimage of a first side of the sample. The images captured by the secondcamera pair 19 are used to create a 3D image of the second side of thesample. In one example, data included in the captured 2D images arecombined into a new dataset and missing information is added to completethe 3D information of the object: depth (distance). By usingtriangulation on matching pixels of the multiple 2D images captured bythe in-flight 3D inspector, the depth component is derived and added tothe dataset. This new dataset describes the object in 3D. This datasetis then used by advanced mathematical algorithms to describe thecharacteristics of the objects. The 3D images of the first and secondsides of the sample are combined to create a 3D image of the entiresample. Once the 3D image of the entire sample is constructed, the 3Dimage can be analyzed to determine if various types of defects arepresent on the sample. For example, if the 3D image does not match apredetermined shape within a specified tolerance, then the sample isdetermined to be defective with respect to shape. In another example, ifthe 3D image shows a flat surface greater than a specified area, thenthe sample is determined to be defective with respect to surfacecontour.

Once the defect information is determined based on the 3D image of thesample, the defect information is stored with the 3D image. The defectinformation can be displayed on display 2 to a user of the in-flight 3Dinspector. The defect information can also be used to generate a reportindicating the number of defects detected across a multiple samples thathave been inspected. The defect information for each sample can be usedby a sorting machine attached to the collector bin 11 of the in-flight3D inspector to determine how the sample is to be routed. The defectinformation for multiple samples can be used to generate a qualityreport indicating the quality grade of the multiple samples.

Various calibrations of the cameras may be performed. An internalcalibration may be performed for each camera. Internal calibrationincludes calibration of principle points, focal lengths, pixel sizeratios, and radial parameters. A stereo calibration may be performed aswell. A stereo calibration addresses the external 3D rotation andtranslation between individual cameras of a stereo system. Aninter-stereo calibration may also be performed to address the external3D rotation and translation between the two stereo systems. In aninter-stereo calibration, a transformation is performed that stitchestwo different side reconstructions into one 3D model.

Capturing Images of Multiple samples in a Single Image

The single sample chute 13 illustrated in FIG. 5 illustrates oneembodiment of the present invention. In another embodiment (not shown inFIG. 5 ) the sample chute may be configured to direct multiple samplesthrough the focal plane at the same moment in time. In this embodiment,the sample chute would cause multiple samples to fall through the focalplane along a single axis at the same time. Aligning the samples along asingle axis prevents one sample from blocking a camera's view of anothersample. The first and second camera pairs would then capture an imageincluding multiple samples instead of just one. Said another way, asingle image would include multiple samples instead of just one. Oncethe images of the multiple samples are captured, the computer system 12would (i) determine which portions of each image are of each sample, and(ii) only use the portions of each image that are of the same sample togenerate the 3D image of the sample.

This configuration would greatly accelerate the rate at which thein-flight 3D inspector can capture images of multiple samples. Forexample, if the sample chute directed ten samples through the focalplane as the same time instead of only one sample, then the in-flight 3Dinspector would be able to collect images of samples ten times faster.Said another way, the in-flight 3D inspector would only requireone-tenth the amount of time to collect images of a set of samples.

FIG. 11 is a flowchart 200 of an in-flight 3D inspector. In step 201, asample is propelled through a focal plane of a dual stereo camerasystem. In step 202, a trigger signal is generated. The trigger signalindicates when the sample will travel through the focal plane of thestereo camera system. In step 203, a predetermined amount of time afterthe trigger signal is generated, an image of the sample is captured byeach camera included in the dual stereo camera system. The sample isilluminated by a light source while the image of the sample is captured.In step 204, the sample is collected in a collector bin and the capturedimages are stored in a memory device.

FIG. 12 is a flowchart 300 of an in-flight 3D inspector with defectprocessing. In step 301, a sample is propelled through a focal plane ofa dual stereo camera system. In step 302, a trigger signal is generated.The trigger signal indicates when the sample will travel through thefocal plane of the stereo camera system. In step 303, a predeterminedamount of time after the trigger signal is generated, an image of thesample is captured by each camera included in the dual stereo camerasystem. The sample is illuminated by a light source while the image ofthe sample is captured. In step 304, the sample is collected in acollector bin and the captured images are stored in a memory device. Instep 305, the captured images are stitched together to generate a 3Dimage of the sample. In step 306, the 3D image of the sample is used todetermine one or more characteristics of the sample.

Various Numbers of Cameras can be Used

The two pairs of cameras 18-19 discussed above are used in a firstembodiment of the present invention. In other embodiments, various othernumbers of cameras may be used. For example, in another embodiment, thein-flight 3D inspector may include only one pair of stereo cameras thatcapture two images of the sample, and the images are used to construct a3D image of the sample from only one point of view. In anotherembodiment, three pairs of stereo cameras can be used to capture siximages of the sample and the images are used to construct a 3D image ofthe sample from three points of view. After review of this disclosure,the reader will appreciate that additional cameras will provideadditional accuracy of the 3D image created by the in-flight 3Dinspector.

Inspection Device Controlled Processing Line System

FIG. 13 is a diagram of an inspection device 400. Inspection device 400includes a processor 401, a storage device 402, an interface circuit403, an optical device 404 and/or other sensors 405. The various partsof inspection device 400 communicate with each other across a bus 406.On skilled in the art will note that various known bus architectures canbe used to implement inspection device 400. One example of a busarchitecture is Peripheral Component Interconnect Express (PCIe), whichprovides standardized communication between various device components.However, many other possible options exist, such as: Ethernet forControl Automation Technology (EtherCAT), Ethernet Industrial Protocol(EtherNet/IP), Process Field Net (PROFINET), Ethernet Powerlink, ThirdGeneration of the Sercos Interface (SERCOS III), Control andCommunication Link (CC-Link IE), and Modbus/TCP, Modbus, Sinec H1,Process Field Bus (Profibus), Controller Area Network Protocol(CANopen), DeviceNet, and FOUNDATION Fieldbus. One example of aprocessor is an intel x86 processor. One example of a storage device isa NAND flash based solid state drive. One example of an interfacecircuit is a Network Interface Card (NIC) that communicates across aphysically connected cable to a network switch or router. Anotherexample of an interface circuit is a Wireless Network InterfaceController (WNIC) that communicates across standards such as WiFi(802.11 protocols), Bluetooth and other such protocols. Another exampleof an interface circuit is a cellular communication device thatcommunicates across cellular networks that use protocols such as GSM,WCDMA, CDMA2000, LTE, etc. All of the above mentioned communicationmethods may be used to implement a data port. An example of an opticaldevice is a high shutter speed, high resolution digital camera that iscontrollable by a computer across a standardized data port, such as USB.Other examples of optical devices include, but are not limited to,millimeter wave cameras, Near-Infr-Red (NIR) cameras, hyper-spectralcameras, and x-ray cameras. Other sensors 405 may include audio,electromagnetic, and odor sensors that are controllable by a computeracross a standardized bus, such as USB. Other examples of sensorsinclude, but are not limited to weight scale sensors, proximity sensors,temperature sensors, humidity sensors, texture sensors, and moisturesensors.

FIG. 14 illustrates an inspection data communication system. Theinspection data communication can be between inspection device 412 andupstream slave device 411 or between inspection device 412 anddownstream slave device 413. The term upstream indicates that samplepass through the slave device before passing through the inspectiondevice 412. The term downstream indicates that samples pass through theinspection device 412 before passing through the salve device.

It is noted herein that a slave device is any device located along thesample processing line. Examples of a slave devices includes, but is notlimited to: a sorting device, a mixing device, a display device, asizing device, a blanching device, a feeding device, a cutting, aslicing device, a baking device, a drying device, a freezing device, acoating device, a washing device.

In one scenario, a sample passes through the slave device 411 and thenpasses through the inspection device 412. Within the inspection device412, the optical device 404 of the inspection device 400 is triggered bythe processor 401 to capture an image. The triggering by the processor401 is executed when a sample is within the field of view of the opticaldevice 404. The image captured by the optical device 404 is then storedinto storage device 402. The processor 401 then processes the capturedimage and determines one or more quality characteristics of the samplein the captured image. Many different quality characteristics may bedetermined from the captured image. Some examples of possible qualitycharacteristics includes, but are not limited to: shape quality (basedon matching a predetermined shape within a specified tolerance, then thesample is determined to be defective with respect to shape), surfacecontour quality (when a flat surface is greater than a specified area,then the sample is determined to be defective with respect to surfacecontour), hole quality (presence of holes in the sample), pest quality(presence of insects in/or on the sample), color quality (irregularcolor of the sample), size quality (irregular size of the sample),moisture level, oil content, fat content, and mycotoxin content. In oneexample, a group of quality characteristics are referred to asinspection data 415. FIG. 18 , FIG. 19 , and FIG. 20 illustrate variousexamples of inspection data. Communication medium 417 can be a wiredmedium such as Ethernet or RS-232. Alternatively, communication medium417 can be wireless medium such as WiFi (802.11) or a cellular link. Theinspection data 415 is then communicated to slave device 411. In thisfashion, the slave device 411 can then analyze the inspection data andadjust the operation of slave device 411 such that more desirablesamples are output from slave device 411. This scenario requires thatslave device 411 include some local knowledge and processing capabilityto analyze the received inspection data and to adjust the operations ofthe slave device 411 based on the analysis.

It is noted herein, the inspection device 400 illustrated in FIG. 13 isonly one example of an inspection device. Another example of aninspection device is the in-flight 3D inspector 1 illustrated in FIGS.1-5 .

It is also noted herein that multiple samples may be within the field ofview of the optical device 404 when an image is captured and thereforequality characteristics of multiple samples may be determined using asingle captured image.

In another scenario, a sample passes through the inspection device 412and then passes through the slave device 413. Within the inspectiondevice 412, the optical device 404 of the inspection device 400 istriggered by the processor 401 to capture an image. The triggering bythe processor 401 is executed when a sample is within the field of viewof the optical device 404. The image captured by the optical device 404is then stored into storage device 402. The processor 401 then processesthe captured image and determines one or more quality characteristics ofthe sample in the captured image. Many different quality characteristicsmay be determined from the captured image. In one example, multiplequality characteristics are referred to as inspection data 415. Theinspection data 415 is then communicated to slave device 413 viacommunication medium 417. Communication medium 417 can be a wired mediumsuch as Ethernet or RS-232. Alternatively, communication medium can bewireless medium such as WiFi (802.11) or cellular link In this fashion,the slave device 413 can then analyze the inspection data and adjust theoperation of slave device 413 such that more desirable samples areoutput from slave device 413. This scenario requires that slave device413 include some local knowledge and processing capability to analyzethe received inspection data and to adjust the operations of the slavedevice 413 based on the analysis.

While the scenario illustrated in FIG. 14 provides the slave devices 411and 413 with the most control over how they operate, in many instancesslave devices 411 and 413 will not have the necessary knowledge andprocessing power to analyze the inspection data generated by theinspection device 412. This problem is addressed by moving theprocessing of the inspection data to the inspection device 412. Thissolution is illustrated in FIG. 15 .

FIG. 15 illustrates a command communication system. The term upstreamindicates that sample pass through the slave device before passingthrough the inspection device 422. The term downstream indicates thatsamples pass through the inspection device 422 before passing throughthe slave device. In this system, a sample passes through the slavedevice 411 and then passes through the inspection device 412. Within theinspection device 412, the optical device 404 of the inspection device400 is triggered by the processor 401 to capture an image. Thetriggering by the processor 401 is executed when a sample is within thefield of view of the optical device 404. The image captured by theoptical device 404 is then stored into storage device 402. The processor401 then processes the captured image and determines one or more qualitycharacteristics of the sample in the captured image. In one example,multiple quality characteristics are referred to as inspection data.Instead of communicating the raw inspection data to the slave device421, the inspection device 422 performs the analysis of the inspectiondata and generates a command 425 to adjust the operation of slave device421. FIG. 21 , FIG. 22 , and FIG. 23 illustrate various examples ofcommands that are generated based on inspection data. For example, acommand may be to set a threshold value to be used by a slave device. Inanother example, a command may be to set a mixing ratio value in a slavedevice. In yet another example, the command may be to adjust a set-pointvalue in a slave device. The command 425 is then communicated to slavedevice 421 via communication medium 427. Slave device 421 then adjustsoperation as commanded such that more desirable samples are output fromslave device 421. This scenario does not require that slave device 421include some local knowledge and processing capability to analyzeinspection data and to adjust the operations of the slave device 421based on the analysis. Rather, this scenario does not require any localknowledge or processing capability to be present on the slave device421, because all the necessary analysis is performed by the inspectiondevice 422. Slave device 421 can operate as a “dumb” terminal thatsimply adjusts operation based on received commands from the inspectiondevice 422. This solution may be very valuable as it reduces the numberof devices that are required to have local processing capability andknowledge, which in turn reduces the cost of the overall system.

In another scenario, a sample passes through the inspection device 422and then passes through the slave device 423. Within the inspectiondevice 412, the optical device 404 of the inspection device 400 istriggered by the processor 401 to capture an image. The triggering bythe processor 401 is executed when a sample is within the field of viewof the optical device 404. The image captured by the optical device 404is then stored into storage device 402. The processor 401 then processesthe captured image and determines one or more quality characteristics ofthe sample in the captured image. In one example, multiple qualitycharacteristics are referred to as inspection data. Instead ofcommunicating the raw inspection data to the slave device 423, theinspection device 422 performs the analysis of the inspection data andgenerates a command 426 to adjust the operation of slave device 423. Thecommand 426 is then communicated to slave device 423 via a communicationmedium. Slave device 423 then adjusts operation as commanded such thatmore desirable samples are output from slave device 423. This scenariodoes not require that slave device 423 include some local knowledge andprocessing capability to analyze inspection data and to adjust theoperations of the slave device 423 based on the analysis. Rather, thisscenario does not require any local knowledge or processing capabilityto be present on the slave device 423, because all the necessaryanalysis is performed by the inspection device 422. Slave device 423 canoperate as a “dumb” terminal that simply adjusts operation based onreceived commands from the inspection device 422. This solution may bevery valuable as it reduces the number of devices that are required tohave local processing capability and knowledge, which in turn reducesthe cost of the overall system.

While the scenario illustrated in FIG. 15 provides cost saving by onlyrequiring a single device in the system to have the necessary knowledgeand processing power, it may be even more advantageous if the none ofthe devices in the system are required to have local processingcapability and knowledge to analyze the captured images. FIG. 16illustrates an inspection data control system using a remote computingdevice.

FIG. 16 illustrates an inspection data control system using a remotecomputing device. The term upstream indicates that sample pass throughthe slave device before passing through the inspection device 432. Theterm downstream indicates that samples pass through the inspectiondevice 432 before passing through the salve device. In this system, asample passes through the slave device 431 and then passes through theinspection device 432. Within the inspection device 432, the opticaldevice 404 of the inspection device 400 is triggered by the processor401 to capture an image 438. The triggering by the processor 401 isexecuted when a sample is within the field of view of the optical device404. The image 438 captured by the optical device 404 is then storedinto storage device 402. The processor 401 does not process the capturedimage 438 to determine one or more quality characteristics of the samplein the captured image 438. Rather, the inspection device 432communicates the captured image 438 to a remote computing device 434. Inone example, remote computing device 434 is a remote computer or serverthat is not part of any machine through which the sample flows. Inresponse to receiving the captured image 438, the remote computingdevice 434 performs the analysis of the captured image 438 and generatesa command 436 to adjust the operation of slave device 431. The command436 is then communicated to slave device 431 via communication medium437. Slave device 431 then adjusts operation as commanded such that moredesirable samples are output from slave device 431. This scenario doesnot require any local knowledge or processing capability to be presenton the slave device 431, because all the necessary analysis is performedby the remote computing device 434. Likewise, this scenario does notrequire any local knowledge or processing capability to be present onthe inspection device 432, because all the necessary analysis isperformed by the remote computing device 434. Both slave device 431 andinspection device 432 can operate as “dumb” terminals that simply adjustoperation based on received commands from the remote computing device434. This solution may be very valuable as it does not require anydevices through which the sample passes to have local processingcapability and knowledge, which in turn reduces the cost of the overallsystem.

In another scenario, a sample passes through the inspection device 432and then passes through the slave device 433. Within the inspectiondevice 432, the optical device 404 of the inspection device 400 istriggered by the processor 401 to capture an image 438. The triggeringby the processor 401 is executed when a sample is within the field ofview of the optical device 404. The image 438 captured by the opticaldevice 404 is then stored into storage device 402. The processor 401does not process the captured image 438 to determine one or more qualitycharacteristics of the sample in the captured image. Rather, theinspection device 432 communicates the captured image 438 to a remotecomputing device 434. In one example, remote computing device 434 is aremote computer or server that is not part of any machine through whichthe sample flows. In response to receiving the captured image 438, theremote computing device 434 performs the analysis of the captured image438 and generates a command 437 to adjust the operation of slave device433. The command 437 is then communicated to slave device 433 viacommunication medium. Slave device 433 then adjusts operation ascommanded such that more desirable samples are output from slave device433. This scenario does not require any local knowledge or processingcapability to be present on the slave device 433, because all thenecessary analysis is performed by the remote computing device 434.Likewise, this scenario does not require any local knowledge orprocessing capability to be present on the inspection device 432,because all the necessary analysis is performed by the remote computingdevice 434. Both slave device 433 and inspection device 432 can operateas “dumb” terminals that simply adjust operation based on receivedcommands from the remote computing device 434. This solution may be veryvaluable as it does not require any devices through which the samplepasses to have local processing capability and knowledge, which in turnreduces the cost of the overall system.

In another example, captured image 438 is not communicated from theinspection device 432 to the remote computing device 434, but ratherinspection data 435 is communicated from the inspection device 432 toremote computing device 434. In this scenario, the inspection device 432captures an image of the sample and from the captured image determinesquality characteristic(s) of the sample. The inspection data (groupingof quality characteristics) is then communicated to the remote computingdevice 434. In response to receiving the inspection data, the remotecomputing device 434 generates one or more commands to adjust one ormore slave devices. In this example, the inspection device 432 requiresthe processing capability to determine the quality characteristics butdoes not require the capability to determine commands for adjustingslave devices.

While the scenario illustrated in FIG. 16 a great improvement, a remotecomputing device can be used in an even more beneficial way. Thisimproved use is illustrated in FIG. 17 .

FIG. 17 illustrates an inspection data control system of multipleprocessing lines using a remote computing device. Each processing line441-446 includes at least one inspection device that is capable ofcapturing an image and sending the capture image and/or inspection databased on the captured image to a remote computing device 440.

The in response to receiving only the captured image data 448, theremote computing device 440 determines quality characteristics and thenbased on those quality characteristics (“inspection data”) the remotecomputing device 440 generates command(s) to adjust the operation ofslave device(s) in the processing line from which the image wascaptured.

In response to receiving the inspection data 447, the remote computingdevice 440 generates command(s) to adjust the operation of slavedevice(s) in the processing line from which the image was captured.

This scenario also reduces the complication of managing multiple sampleprocessing lines. A single remote computing device 440 could receiveinspection data from various inspection devices included in variousprocessing lines 441-446. In this fashion, the single remote computingdevice 440 could monitor and adjust all the various slave devices inprocessing lines 441-446. This scenario can also provide for advancedlearning because all inspection data from all processing lines 441-446are received by the remote computing device 440, which in turn allowsfor improved artificial intelligence learning by way of access to largersets of relevant inspection data.

This scenario also allows for real-time monitoring and adjusting ofmultiple processing lines located at various locations around the world.

FIG. 24 is a flowchart 300 of an inspection data communication system.In step 301, an image of a sample is captured by an inspection device asthe sample travels along a processing line. In step 302, the capturedimage is processed with respect to quality characteristic(s) andinspection data is generated. In step 303, the inspection data iscommunicated from the inspection device to another device located alongthe sample processing line. In step 304, the device receives theinspection data and in response adjusts the operation of the devicebased at least in part on the inspection data received.

FIG. 25 is a flowchart 400 of a command communication system. In step401, an image of a sample is captured by an inspection device as thesample travels along a processing line. In step 402, the captured imageis processed with respect to quality characteristic(s) and inspectiondata is generated. In step 403, a command is generated based at least inpart on the inspection data. In step 404, the command is thencommunicated from the inspection device to another device located alongthe sample processing line. In step 405, the device receives the commandand in response adjusts the operation of the device based at least inpart on the command received.

FIG. 26 is a flowchart 500 of an inspection data control system using aremote computing device. In step 501, an image of a sample is capturedby the inspection device as the sample travels along a processing line.In step 502, the captured image is communicated to the remote computingdevice. In step 503, in response to receiving the captured image, theremote computing device determines quality characteristic(s) andgenerates inspection data. In step 504, the remote computing deviceprocesses the inspection data and generates a command. In step 505, thecommand is communicated from the remote computing device to a devicelocated along the sample processing line. In step 506, the devicereceives the command and in response adjusts the operation of the devicebased at least in part on the command received.

FIG. 27 is a flowchart 600 of an inspection data control system using aremote computing device. In step 601, an image of a sample is capturedby the inspection device as the sample travels along the processingline. In step 602, the captured image is processed with respect toquality characteristic(s) and inspection data is generated. In step 603,the inspection data is communicated from the inspection device to aremote computing device and in response to receiving the inspectiondata, the remote computing device processes the inspection data andgenerates a command. In step 604, the command is communicated from theremote computing device to another device located along the sampleprocessing line. In step 605, the device receives the command and inresponse adjusts the operation of the device based at least in part onthe command received.

Distributed Ledger

A distributed ledger, sometimes referred to as “Blockchain”, is a sharedpublic ledger on which a system relies. The distributed ledger has alinked list data structure, with each block containing a hash of theprevious block. Each block is formed by a proof-of-work algorithm,through which consensus of this distributed ledger could be obtained viathe longest possible chain. The distributed ledger provides the basisfor a distributed system that does not require trust between varioususers and is extendable in many ways through modifications of theparameters of the chain.

A block is an aggregated set of data. Data are collected and processedto fit in a block through a process called mining. Each block could beidentified using a cryptographic hash (also known as a digitalfingerprint). The formed block will contain a hash of the previousblock, so that blocks can form a chain from the first block ever (knownas the Genesis Block) to the newly formed block. In this way, all thedata could be connected via a linked list structure.

Data are contained inside blocks as well as an arbitrary integer (callednounce) that is necessary for producing the proof-of-work. In theexample of Bitcoin, a block contains a header and relevant transactiondata. A merkle tree of transactions is created and the hash of the rootis included in the header. A merkle tree is a full binary tree of a hashvalues. At the bottom level of the tree, each transaction has a nodecontaining its hash value. The tree is constructed in a way such thatthe parent node has a value of the hash of the data contained in itschildren concatenating together. The merkle tree data structure allowsfast validation by constructing a merkle tree path from the bottom levelof the tree up to the root node. Since each bitcoin transaction outputcan be spent only once, as long as the output is spent, it could beerased out of the tree structure using some pruning algorithms. In thisway, disk usage is reduced while the validation functions are preserved.

Various blocks in the blockchain are connected to specific other blocksin the blockchain. In one example, each block contains a hash of itsprevious block. In bitcoin blockchain for example, the block header hasa field for previous block hash. Therefore, all blocks will contain areference of its previous block thereby enabling the chain to be buildup to the genesis block.

A fork on the block chain may occur. A fork in the block chain occursdue to two blocks computed at a very short time interval. The subsequentblocks may build upon both blocks and both of the chains remain valid.In subsequent process of mining, one fork would be longer than the otherfork, in this case, the longer chain would be accepted by the networkand the short would not be used unless its length exceeds the longerchain in the future.

Many distributed ledgers, such as Bitcoin blockchain, use aproof-of-work algorithm for reaching a consensus. The cryptographic hashfunction of each block must be smaller than a specific value in order tobe considered value. A nonce is therefore included in the block for thisfeature. By using the proof-of-work method, in order to change the datain one block, all successors of that block must be re-written and a hugeamount of calculation is necessary. In addition, the longest chain wouldbe accepted by the network whereas the shorter ones would be discardedat the situation of branches of the chain. This makes the data in blockspractically unmodifiable. Further, the more blocks that are built uponthe block in which the data is contained, more processing is required tooverwrite the data.

However, the blockchain may use other methods of consensus. For example,a blockchain may use Scrypt for proof-of-work algorithm instead of hashfunctions. In addition, the blockchain could be extended for scientificcomputation where a correct solution to a certain problem could act avalidation method. In this way, the computation power may be used tohelp solving scientific problems and contribute to scientific research.

In a distributed ledger, each user running a full node on the computerwill download a full copy of the whole blockchain, which will includedata of all events, such as transactions, recorded on the blockchain.After that, each node can run independently to process any incomingevents, such as transactions, and propagate the event further. The nodecan also contribute to the establishment of the consensus by mining toinclude event data in a block and then to find a proof-of-work for theblock. There is not a central node processing the data and distributingthe data, rather every node can run independently and broadcast any workthat is proved. This model of distributed computation could be extendedto many other services such as Domain Name Server.

Quality Inspection Data Distributed Ledger

While distributed ledgers have been utilized to perform financialtransactions, distributed ledgers have not been utilized to performrecordation and distribution of quality inspection data.

Quality inspection data is measured and recorded for many differentitems around the world, such as pharmaceuticals, mechanical hardware,agricultural foods and many, many more.

One challenge is the acquisition of quality inspection data. Forexample, some quality inspection data is generated by humans reviewingitems manually. This process is prone to large variances depending onthe human conducting the inspection as well as the state of the humanwhen the inspection is conducted. In other examples, computer automationis used to help, or entirely, acquire the quality inspection data asdisclosed above.

Regardless of the method in which the quality inspection data isacquired, a second challenge is the integrity of the quality data thatis reported to interested parties, such as owners, purchasers,manufacturers, etc. For example, in the almond industry many purchasersare weary of the quality data that is alleged by various almondproviders. The uncertainty spawns from various sources. First, was thequality inspection data reliable? Second, was the quality inspectiondata accurately managed and is it accurately aligned with the productbeing offered? Third, was the quality inspection data intentionallytampered with to increase the market price of the product being sold?Fourth, difficulty to gain access to the data regardless of the threeconcerns listed above. All of these uncertainties lead to time and costinefficiencies. A trust worthy, reliable and cost efficient solution isprovided herein.

Regarding the reliability of the quality inspection data, as discussedabove, an automated system such as the in-flight optical inspector canbe used to acquire reliable and consistent quality inspection data.

Regarding the management, possible tampering, and access to the acquiredquality inspection data a new quality inspection data distributed ledgeris disclosed. This quality inspection data distributed ledger does notperform financial transactions. Rather, the quality inspection datadistributed ledger validates the source, timing, product association,and integrity of the quality inspection data.

FIG. 28 is a diagram of a quality inspection data distributed ledgerflowchart 620. In step 621, one or more samples to be inspected arecollected. In step 622, the one or more samples are assigned a uniqueidentification code. In one example, the identification code is affixedto a container containing the one or more samples. In another example,the code is affixed to the one or more samples themselves. The code maybe communicated by use of a Quick Response (QR) code, a bar code,printed text, Radio Frequency Identification (RFID) tag, human manualentry, a Near Field Communication (NFC) signal, a token, or any othermethod of communication known to one of skill in the art.

Once the identification code has been assigned to the one or moresamples, the one or more samples are inspected in step 623. Thisinspection can be performed by any possible method. In one example, theinspection can be performed by human inspection. In another example, theinspection can be performed by an automated inspection. In yet anotherexample, the inspection can be performed by an in-flight 3D inspector asdisclosed above.

Upon completion of the inspection of the one or more samples, in step624 the resulting inspection data and the identification code arewritten into a new quality inspection data block. An example of aquality inspection data block is illustrated in FIG. 29 . The exemplaryquality inspection data block of FIG. 29 may include the following datafields: an inspection entity that conducted the inspection, inspectionlocation where the inspection was conducted, the sensor identificationnumber that identifies the sensor or inspection device that performedthe inspection, the lot number of the one or more samples, the totalweight of the lot of samples, analysis completion timestamp indicatingwhen the inspection was performed or completed, the amount of productanalyzed (for example in weight or quantity), the moisture content ofthe samples, the kernel size of the samples, the uniformity ratio of thesamples (average, median, variance, etc.), the percentage or number of“good” samples or samples that pass all required characteristics, thepercentage or number of dissimilar samples, the percentage or number ofchipped and scratched samples, the percentage or number of samplesincluding another type of defect, the percentage or number of samplesthat have serious damage, and the quality grade of the one or moresamples, such as U.S. Extra #1 grade.

The exemplary quality inspection data block of FIG. 29 may also includethe following data fields: a color value, a microtoxin value(milligram/kg, microgram/kg, Parts Per Million, Parts Per Billion . . .), a temperature value, an acidity (pH) value, a pressure value (kPA,PSI . . . ), a volume per unit time (cubic meters per second), an amountof discolored product (number of percentage), an amount of brokenproduct (number or percentage), an amount of rancid product (number orpercentage), an amount of moldy product (number of percentage), anamount of immature product (number or percentage), an amount of unripeproduct (number or percentage), or an amount of rotten soft product(number or percentage).

The inspection data fields listed above are only provided to beexemplary. One skilled in the art will appreciate that any othercharacteristic determined during inspection can be included in thequality inspection data block. Likewise, any of the inspection datafields listed can be omitted from the quality inspection data block aswell. A list of other possible inspection devices is listed below.

-   -   Optical sensors    -   Moisture sensors    -   Microtoxin sensors    -   Thermometer sensors    -   Acidity (pH) sensors    -   Microwave sensors    -   Pressure sensors    -   Level sensors    -   Ultrasonic sensors    -   Flow sensors    -   Viscosity sensors    -   Conductance/Impedance sensors    -   Electronic Nose (sniffing) sensors    -   X-ray sensors    -   Multi Spectral (visual/non visual) sensors    -   Weight sensors    -   Refractometers sensors    -   Tenderometer sensors    -   Firmness sensors    -   Hardness sensors    -   Proximity sensor

The quality inspection data block may also include hash information. Thequality inspection data block may include any of the following hashinformation: Hash of the quality inspection data block itself, hash ofthe previous quality inspection data block, hash of the next qualityinspection data block, or a Merkle root.

In the example where an inspector with computational capabilitiesperforms the inspection, the inspector may create the quality inspectiondata block itself upon completion of the inspection process. In theexample where the inspector does not have computational capabilities,the data collected by the inspector can be manually entered intocomputationally capable device to create the quality inspection datablock.

In step 625, after the quality inspection data block is created, thequality inspection data block is added to the distributed ledger. In oneexample, the distributed ledger is referred to as a blockchain. In theexample where an inspector includes networking capabilities, theinspector can add the new quality inspection data block to thedistributed ledger via a network. Once the quality inspection data blockhas been added to the distributed ledger, the quality inspection datablock is available for viewing by anyone on the network and cannoteasily be changed.

This quality inspection data distributed ledger will solve the problemscurrently facing the consumers of quality inspection data. Consumers ofquality inspection data will now have a single source of trustworthyquality inspection data that is easy to access.

Adaptable Inspection Unit & Adaptable Sorter Unit

Processing lines are widely used to inspect and sort large quantities ofa specific item. For example, processing lines are used to inspect andsort eatable items such as fruits and nuts. Alternatively, processinglines are used to inspect and sort pharmaceutical pills. A popularexample of a simple processing line is a conveyor processing line whereitems are propelled through the processing line via a conveyor belt thatis wound around the conveyor head pulley and tail pulley. Other examplesof processing lines include, but are not limited to, a flume, a rollerbelt, a shaker (conventional and linear motion), a slide, a chute, aconveyor tube, a bucket elevator, and a screw conveyor. To date, basicprocessing lines, such as conveyors, have not been adaptable to workwith any improved processing devices.

Recent improvements in the area of sample inspection and sorting, as aredisclosed above in the present application, have provided tremendousimprovements in the areas of reliability of quality inspection data,high accuracy quality inspection data, low cost of quality inspectiondata, as well as intelligent and automatic dynamic control of sortingdevices. Currently, owners and operators of legacy simple processinglines, such as conveyors and chutes, cannot attain these improvementswithout upgrading their entire processing line without suffering largecosts, new process planning and time delays. A solution is needed toprovide these improved inspection and sorting technologies in a way thatcan be easily and cost effectively adapted to a legacy simple processingline. A solution to this need is provided herein.

FIG. 30 is a diagram of a conveyor for manual inspection or sorting.This system of using a conveyor for manual inspection and sorting isused widely around the world. In operation, sample 701 is caused tobecome in contact with conveyor 700. Upon contact, sample 701 is movedvia a rotating conveyor belt of conveyor 700. A conveyor belt usuallyrotates the conveyor belt about two or more pulleys. The frictionbetween the sample and the rotating conveyor belt causes the sample tobe move along the direction of the conveyor belt movement. A human islocated proximate to the conveyor where the human can see the sample asit travels past the human. In the event that the human needs to moreclearly see the sample, the human can pick up and more carefully inspectthe sample. The human has the responsibility of determining the qualityof the sample. Further, the human has the responsibility to determinehow the sample should be sorted based on the determined quality. Forexample, the human may determine that the sample is of a quality thatshould be discarded. In which case, the human would manually with thehuman's hand, or with a tool manually manipulated by the human, causethe sample to be removed from the processing line and sent to a group ofdiscarded samples. In another example, the human may determine that thesample is of a mediocre quality that should not be discarded, but alsoshould not be grouped with top quality samples. In which case, the humanwould manually cause the sample to be moved to a group of mediocresamples. In yet another example, the human may determine that the sampleis of top quality. In which case, the human would not cause the sampleto be moved at all but would rather allow the sample to continue throughthe processing line to be grouped with all other top quality samples.The samples may be fruits, nuts, pills or any other type of item forwhich quality control is required.

This method requires a large amount of human attention and time.Moreover, this method of manual inspection and sorting is prone to humanerror, low quality accuracy, and low repeatability (inconsistentresults).

FIG. 31 is a diagram of a conveyor with an adaptable inspection unitattached to the conveyor. Conveyor 710 is the same as conveyor 700,except in that adaptable inspection unit 712 has been physically mountedto conveyor 700. Similar to FIG. 30 , in operation, sample 711 is causedto become in contact with conveyor 710. Upon contact, sample 711 ismoved via a rotating conveyor belt of conveyor 710. The friction betweenthe sample and the rotating conveyor belt causes the sample to be movealong the direction of the conveyor belt movement.

The adaptable inspection unit 712 is attached to the conveyor 710 viaone or more mounting brackets 713. One skilled in the art will readilyrealize that a various number of brackets and various styles of bracketscan be used to mount the adaptable inspection unit 712 to conveyor 710.Mounting Bracket 713 can attach to either the adaptable inspection unit712 or the conveyor 710 using various items, such as bolts, screws,pins, locks, clamps, welds (metals or thermoplastics), adhesive, slots,magnets, rails, gravity or friction.

The adaptable inspection unit 712 includes an attachment mechanism, aninspection sensor device (optical receiver), a data port and a powerport. The data port and the power port may be combined into a singlephysical port that connects to a single cable 714 that includes bothpower conductors and data conductors. Alternatively, the adaptableinspection unit 712 may include a data port that is separate from thepower port. Further, the adaptable inspection unit 712 may include anantenna connectable data port that connects to an antenna 715 so toallow for wireless communication. FIG. 31 does not illustrate theinspection sensor device. FIG. 39 illustrates a block diagram of anadaptable inspection unit 790 that includes an attachment mechanism 791,an inspection sensor device 792, a data port 793, and a power port 794.

In operation, the conveyor 710 causes the sample 711 to travel under theadaptable inspection unit 712. While the sample is in view of theinspection sensor device that is included in the adaptable inspectionunit 712 one or more images of the sample are captured and stored in amemory device. The memory device may be included in the adaptableinspection unit 712 or may be included in a device that communicateswith the adaptable inspection unit 712 via the data port (wired orwireless). The captured sensor data (e.g., images) are then processed bya processor executing a quality inspection algorithm. In one example,the adaptable inspection unit 712 includes the 3D inspector described indetail above. In another example, a 3D image of the sample is generatedbased on the one or more images captured by the adaptable inspectionunit 712. In yet another example, the captured 2D image is used toperform the inspection. The 3D or 2D image(s) are used to determine aquality characteristic of the sample. In one example, the qualitycharacteristic is generated by the adaptable inspection unit 712 andoutput via the data port. In another example, the one or more capturedimages are output from the adaptable inspection unit 712 to anotherdevice that determines the quality characteristics of the sample.Adaptable inspection unit 712 provides improve quality inspectioncompared to unreliable inspection by human eyes without the cost ofreplacing an entire processing line. Moreover, adaptable inspection unit712 is able to inspect many more samples per unit time than could beinspected by a human.

In the example of FIG. 31 , all samples are directed toward the samelocation regardless of measured quality because there is no sortingfunctionality attached to the conveyor 710.

FIG. 32 is a diagram of a conveyor with an adaptable sorting unitattached to the conveyor. Conveyor 720 is the same as conveyor 700,except in that adaptable sorter unit 722 has been physically mounted toconveyor 720. Similar to FIG. 30 , in operation, sample 721 is caused tobecome in contact with conveyor 720. Upon contact, sample 721 is movedvia a rotating conveyor belt of conveyor 720. The friction between thesample and the rotating conveyor belt causes the sample to be move alongthe direction of the conveyor belt movement.

The adaptable sorter unit 722 is attached to the conveyor 720 via one ormore mounting brackets 723. One skilled in the art will readily realizethat a various number of brackets and various styles of brackets can beused to mount the adaptable sorter unit 722 to conveyor 720. MountingBracket 723 can attach to either the adaptable sorter unit 722 or theconveyor 720 using various items, such as bolts, screws, pins, locks,clamps, welds (metals or thermoplastics), adhesive, slots, magnets,rails, gravity, or friction.

The adaptable sorter unit 722 includes an attachment mechanism, asorting device capable of deflecting a sample, a data port, and a powerport. The data port and the power port may be combined into a singlephysical port that connects to a single cable 724 that includes bothpower conductors and data conductors. Alternatively, the adaptablesorter unit 722 may include a data port that is separate from the powerport. Further, the adaptable sorter unit 722 may include an antennaconnectable data port that connects to an antenna 725 so to allow forwireless communication. FIG. 32 does not illustrate the sorting device.FIG. 40 illustrates a block diagram of an adaptable sorter unit 800 thatincludes an attachment mechanism 801, a sorting device 802, a data port803, and a power port 804.

In operation, the conveyor 720 causes the sample 721 to travel under theadaptable sorter unit 722. While the sample is in reach of the sortingdevice that is included in the adaptable sorter unit 722 the sample issorted as instructed. In one example, the sorting instruction isreceived via the data port and stored in a memory included in theadaptable sorting unit 722. In another example, quality characteristicdata is received via the data port and in response the adaptable sorterunit 722 generates the sorting instruction. In yet another example, theinformation received via the data port is a percentage of samples to bedeflected. Communication with the adaptable sorter unit 722 may beperformed via the data port (wired or wireless). The sorting device maybe by a vacuum system, a mechanical pedal system, an air jet system, ora mechanical gate. The adaptable sorting unit 722 performs automatedsorting so that high quality samples are automatically separated fromlow quality samples.

Adaptable sorter unit 722 provides improve sorting compared tounreliable sorting by human hands without the cost of replacing anentire processing line. Moreover, adaptable sorter unit 722 is able tosort many more samples per unit time than could be sorted by a human.

FIG. 33 is a diagram of a conveyor with an adaptable inspection unitattached to the ceiling above the conveyor. Conveyor 730 is the same asconveyor 700, except in that adaptable inspection unit 732 has beenphysically mounted to the ceiling above conveyor 730. Similar to FIG. 30, in operation, sample 731 is caused to become in contact with conveyor730. Upon contact, sample 731 is moved via a rotating conveyor belt ofconveyor 730. The friction between the sample and the rotating conveyorbelt causes the sample to be move along the direction of the conveyorbelt movement.

The adaptable inspection unit 732 is attached to the ceiling aboveconveyor 730 via one or more mounting brackets 733. One skilled in theart will readily realize that a various number of brackets and variousstyles of brackets can be used to mount the adaptable inspection unit732 to the ceiling above conveyor 730. Mounting Bracket 733 can attachto either the adaptable inspection unit 712 or the ceiling aboveconveyor 730 using various items, such as bolts, screws, pins, locks,clamps, welds (metals or thermoplastics), adhesive, slots, magnets,rails, gravity, or friction.

The adaptable inspection unit 732 includes an attachment mechanism, aninspection sensor device (optical receiver), a data port and a powerport. The data port and the power port may be combined into a singlephysical port that connects to a single cable 734 that includes bothpower conductors and data conductors. Alternatively, the adaptableinspection unit 732 may include a data port that is separate from thepower port. Further, the adaptable inspection unit 732 may include anantenna connectable data port that connects to an antenna 735 so toallow for wireless communication. FIG. 33 does not illustrate theinspection sensor device. FIG. 39 illustrates a block diagram of anadaptable inspection unit 790 that includes an attachment mechanism 791,an inspection sensor device 792, a data port 793, and a power port 794.

In operation, the conveyor 730 causes the sample 731 to travel under theadaptable inspection unit 732. While the sample is in view, or reach, ofthe inspection sensor device that is included in the adaptableinspection unit 732 one or more characteristics and/or images of thesample are captured and stored in a memory device. The memory device maybe included in the adaptable inspection unit 732 or may be included in adevice that communicates with the adaptable inspection unit 732 via thedata port (wired or wireless). The captured characteristics and/orimage(s) are then processed by a processor executing a qualityinspection algorithm. In one example, the adaptable inspection unit 732includes the 3D inspector described in detail above. In another example,a 3D image of the sample is generated based on the one or more imagescaptured by the adaptable inspection unit 732. In yet another example,the captured 2D image is used to perform the inspection. The 3D and/or2D image(s) are used to determine a quality characteristic of thesample. In one example, the quality characteristic is generated by theadaptable inspection unit 732 and output via the data port. In anotherexample, the one or more captured images are output from the adaptableinspection unit 732 to another device that determines the qualitycharacteristics of the sample. Adaptable inspection unit 732 providesimprove quality inspection compared to unreliable inspection by humaneyes without the cost of replacing an entire processing line. Moreover,adaptable inspection unit 732 is able to inspect many more samples perunit time than could be inspected by a human.

In the example of FIG. 33 , all samples are directed toward the samelocation regardless of measured quality because there is no sortingfunctionality attached to the conveyor 730.

FIG. 34 is a diagram of a conveyor with an adaptable sorting unitattached to a ceiling above the conveyor. Conveyor 740 is the same asconveyor 700, except in that adaptable sorter unit 742 has beenphysically mounted to the ceiling above conveyor 740. Similar to FIG. 30, in operation, sample 741 is caused to become in contact with conveyor740. Upon contact, sample 741 is moved via a rotating conveyor belt ofconveyor 740. The friction between the sample and the rotating conveyorbelt causes the sample to be move along the direction of the conveyorbelt movement.

The adaptable sorter unit 742 is attached to the ceiling above conveyor740 via one or more mounting brackets 743. One skilled in the art willreadily realize that a various number of brackets and various styles ofbrackets can be used to mount the adaptable sorter unit 742 to theceiling above conveyor 740. Mounting Bracket 743 can attach to eitherthe adaptable sorter unit 742 or the ceiling above conveyor 740 usingvarious items, such as bolts, screws, pins, locks, clamps, welds (metalsor thermoplastics), adhesive, slots, magnets, rails, gravity, orfriction.

The adaptable sorter unit 742 includes an attachment mechanism, asorting device capable of deflecting a sample, a data port, and a powerport. The data port and the power port may be combined into a singlephysical port that connects to a single cable 744 that includes bothpower conductors and data conductors. Alternatively, the adaptablesorter unit 742 may include a data port that is separate from the powerport. Further, the adaptable sorter unit 742 may include an antennaconnectable data port that connects to an antenna 745 so to allow forwireless communication. FIG. 34 does not illustrate the sorting device.FIG. 40 illustrates a block diagram of an adaptable sorter unit 800 thatincludes an attachment mechanism 801, a sorting device 802, a data port803, and a power port 804.

In operation, the conveyor 740 causes the sample 741 to travel under theadaptable sorter unit 742. While the sample is in reach of the sortingdevice that is included in the adaptable sorter unit 742 the sample issorted as instructed. In one example, the sorting instruction isreceived via the data port and stored in a memory included in theadaptable sorting unit 742. In another example, quality characteristicdata is received via the data port and in response the adaptable sorterunit 742 generates the sorting instruction. In yet another example, theinformation received via the data port is a percentage of samples to bedeflected. Communication with the adaptable sorter unit 742 may beperformed via the data port (wired or wireless). The sorting device maybe a vacuum system, a mechanical pedal system, an air jet system, or amechanical gate. The adaptable sorting unit 742 performs automatedsorting so that high quality samples are automatically separated fromlow quality samples.

Adaptable sorter unit 742 provides improve sorting compared tounreliable sorting by human hands without the cost of replacing anentire processing line. Moreover, adaptable sorter unit 742 is able tosort many more samples per unit time than could be sorted by a human.

FIG. 35 is a diagram of a conveyor with an adaptable inspection unitattached to a mounting stand. Conveyor 750 is the same as conveyor 700,except in that adaptable inspection unit 752 has been physically mountedto a mounting stand 756 located next to conveyor 750. Similar to FIG. 30, in operation, sample 751 is caused to become in contact with conveyor750. Upon contact, sample 751 is moved via a rotating conveyor belt ofconveyor 750. The friction between the sample and the rotating conveyorbelt causes the sample to be move along the direction of the conveyorbelt movement.

The adaptable inspection unit 752 is attached to the mounting stand 756,located next to conveyor 750, via one or more mounting brackets 753. Oneskilled in the art will readily realize that a various number ofbrackets and various styles of brackets can be used to mount theadaptable inspection unit 752 to the mounting stand 756. MountingBracket 753 can attach to either the adaptable inspection unit 752 orthe mounting stand 756 using various items, such as bolts, screws, pins,locks, clamps, welds (metals or thermoplastics), adhesive, slots,magnets, rails, gravity, or friction.

The adaptable inspection unit 752 includes an attachment mechanism, aninspection sensor device (e.g., optical receiver), a data port and apower port. The data port and the power port may be combined into asingle physical port that connects to a single cable 754 that includesboth power conductors and data conductors. Alternatively, the adaptableinspection unit 752 may include a data port that is separate from thepower port. Further, the adaptable inspection unit 752 may include anantenna connectable data port that connects to an antenna 755 so toallow for wireless communication. FIG. 35 does not illustrate theinspection sensor device. FIG. 39 illustrates a block diagram of anadaptable inspection unit 790 that includes an attachment mechanism 791,an inspection sensor device 792, a data port 793, and a power port 794.

In operation, the conveyor 750 causes the sample 751 to travel under theadaptable inspection unit 752. While the sample is in view, or reach, ofthe inspection sensor device that is included in the adaptableinspection unit 752 one or more characteristics and/or images of thesample are captured and stored in a memory device. The memory device maybe included in the adaptable inspection unit 752 or may be included in adevice that communicates with the adaptable inspection unit 752 via thedata port (wired or wireless). The captured characteristics and/orimage(s) are then processed by a processor executing a qualityinspection algorithm. In one example, the adaptable inspection unit 752includes the 3D inspector described in detail above. In another example,a 3D image of the sample is generated based on the one or more imagescaptured by the adaptable inspection unit 752. In yet another example,the captured 2D image is used to perform the inspection. The 3D and/or2D image(s) are used to determine a quality characteristic of thesample. In one example, the quality characteristic is generated by theadaptable inspection unit 752 and output via the data port. In anotherexample, the one or more captured images are output from the adaptableinspection unit 752 to another device that determines the qualitycharacteristics of the sample. Adaptable inspection unit 752 providesimprove quality inspection compared to unreliable inspection by humaneyes without the cost of replacing an entire processing line. Moreover,adaptable inspection unit 752 is able to inspect many more samples perunit time than could be inspected by a human.

In the example of FIG. 35 , all samples are directed toward the samelocation regardless of measured quality because there is no sortingfunctionality attached to the conveyor 750.

FIG. 36 is a diagram of a conveyor with an adaptable sorting unitattached to a mounting stand. Conveyor 760 is the same as conveyor 700,except in that adaptable sorter unit 762 has been physically mounted toa mounting stand 766 located next to conveyor 760. Similar to FIG. 30 ,in operation, sample 761 is caused to become in contact with conveyor760. Upon contact, sample 761 is moved via a rotating conveyor belt ofconveyor 760. The friction between the sample and the rotating conveyorbelt causes the sample to be move along the direction of the conveyorbelt movement.

The adaptable sorter unit 762 is attached to the mounting stand 766,located next to conveyor 760, via one or more mounting brackets 763. Oneskilled in the art will readily realize that a various number ofbrackets and various styles of brackets can be used to mount theadaptable sorter unit 762 to the mounting stand 766. Mounting Bracket763 can attach to either the adaptable sorter unit 762 or the mountingstand 766 using various items, such as bolts, screws, pins, locks,clamps, welds (metals or thermoplastics), adhesive, slots, magnets,rails, gravity, or friction.

The adaptable sorter unit 762 includes an attachment mechanism, asorting device capable of deflecting a sample, a data port, and a powerport. The data port and the power port may be combined into a singlephysical port that connects to a single cable 764 that includes bothpower conductors and data conductors. Alternatively, the adaptablesorter unit 762 may include a data port that is separate from the powerport. Further, the adaptable sorter unit 762 may include an antennaconnectable data port that connects to an antenna 765 so to allow forwireless communication. FIG. 36 does not illustrate the sorting device.FIG. 40 illustrates a block diagram of an adaptable sorter unit 800 thatincludes an attachment mechanism 801, a sorting device 802, a data port803, and a power port 804.

In operation, the conveyor 760 causes the sample 761 to travel under theadaptable sorter unit 762. While the sample is in reach of the sortingdevice that is included in the adaptable sorter unit 762 the sample issorted as instructed. In one example, the sorting instruction isreceived via the data port and stored in a memory included in theadaptable sorting unit 762. In another example, quality characteristicdata is received via the data port and in response the adaptable sorterunit 762 generates the sorting instruction. In yet another example, theinformation received via the data port is a percentage of samples to bedeflected. Communication with the adaptable sorter unit 762 may beperformed via the data port (wired or wireless). The sorting device maybe a vacuum system, a mechanical pedal system, an air jet system, or amechanical gate. The adaptable sorting unit 762 performs automatedsorting so that high quality samples are automatically separated fromlow quality samples.

Adaptable sorter unit 762 provides improve sorting compared tounreliable sorting by human hands without the cost of replacing anentire processing line. Moreover, adaptable sorter unit 762 is able tosort many more samples per unit time than could be sorted by a human.

FIG. 37 is a diagram of a conveyor with an adaptable inspection unitattached to the conveyor sidewall. The adaptable inspection unit can beattached permanently or temporarily to the conveyor sidewall. Conveyor780 includes one or more sidewalls 781 and a belt that rotates about twoor more pulleys. The sidewall 781 is included in the conveyor 780 so toprevent samples from fall off the sides of the conveyor 780. Thesidewall 781 of the conveyor 780 can be used to support the adaptableinspection unit 782. Although not shown in FIG. 37 , the sidewall 781can also be used to mount an adaptable sorter unit.

The adaptable inspection unit 782 (or an adaptable sorter unit) can beattached using many different mechanisms. Some of these mechanisms arelisted on FIG. 37 . These attachment mechanisms include welding theadaptable inspection unit 782 to the conveyor sidewall 781, gluing theadaptable inspection unit 782 to the conveyor sidewall 781, clamping theadaptable inspection unit 782 to the conveyor sidewall 781, magneticallyattracting the adaptable inspection unit 782 to the conveyor sidewall781, latching the adaptable inspection unit 782 to the conveyor sidewall781, locking the adaptable inspection unit 782 to the conveyor sidewall781, location pinning the adaptable inspection unit 782 to the conveyorsidewall 781, rail mating the adaptable inspection unit 782 to theconveyor sidewall 781, slide fitting the adaptable inspection unit 782to the conveyor sidewall 781, lock pinning the adaptable inspection unit782 to the conveyor sidewall 781, or using gravity and friction to“attach” the adaptable inspection unit 782 to the conveyor sidewall 781.

The adaptable inspection unit 782 includes an attachment mechanism, aninspection sensor device (e.g., an optical receiver), a data port and apower port. The data port and the power port may be combined into asingle physical port that connects to a single cable 784 that includesboth power conductors and data conductors. Alternatively, the adaptableinspection unit 782 may include a data port that is separate from thepower port. Further, the adaptable inspection unit 782 may include anantenna connectable data port that connects to an antenna 785 so toallow for wireless communication. FIG. 37 does not illustrate theinspection sensor device. FIG. 39 illustrates a block diagram of anadaptable inspection unit 790 that includes an attachment mechanism 791,an inspection sensor device 792, a data port 793, and a power port 794.

In operation, the conveyor 780 causes the sample 781 to travel under theadaptable inspection unit 782. While the sample is in view, or reach, ofthe inspection sensor device that is included in the adaptableinspection unit 782 one or more characteristics and/or images of thesample are captured and stored in a memory device. The memory device maybe included in the adaptable inspection unit 782 or may be included in adevice that communicates with the adaptable inspection unit 782 via thedata port (wired or wireless). The captured characteristics and/orimage(s) are then processed by a processor executing a qualityinspection algorithm. In one example, the adaptable inspection unit 782includes the 3D inspector described in detail above. In another example,a 3D image of the sample is generated based on the one or more imagescaptured by the adaptable inspection unit 782. In yet another example,the captured 2D image is used to perform the inspection. The 3D and/or2D image(s) are used to determine a quality characteristic of thesample. In one example, the quality characteristic is generated by theadaptable inspection unit 782 and output via the data port. In anotherexample, the one or more captured images are output from the adaptableinspection unit 782 to another device that determines the qualitycharacteristics of the sample. Adaptable inspection unit 782 providesimprove quality inspection compared to unreliable inspection by humaneyes without the cost of replacing an entire processing line. Moreover,adaptable inspection unit 782 is able to inspect many more samples perunit time than could be inspected by a human.

In the example of FIG. 37 , all samples are directed toward the samelocation regardless of measured quality because there is no sortingfunctionality attached to the conveyor 780. However, an adaptable sorerunit as described above could be placed further down the conveyor 780 toprovide sorting functionality as well as inspection functionality.

FIG. 38 is a diagram of a conveyor with an adaptable inspection unitattached to the conveyor and an adaptable sorting unit attached to theconveyor. As discussed above an adaptable inspection unit 772 and anadaptable sorter unit 776 can be mounted or positioned near an existingprocessing line. With these solutions, both an adaptable inspection unit772 and an adaptable sorter unit 776 can be added to an existingprocessing line to allow for both automated quality inspection ofsamples as well as automated sorting of samples, without the cost ofreplacing the entire processing line. As discussed above, the adaptableinspection unit 772 and the adaptable sorter unit 776 can communicatewith each other in various methods to achieve the desired inspection andsorting functions. Further, the adaptable inspection unit 772 and theadaptable sorter unit 776 can communicate with each in addition to aseparate computing device, such as a network server to achieve thedesired inspection and sorting functions. The drawings and relateddisclosure regarding FIGS. 14-27 illustrate and describe multiplemethods in which the adaptable inspection unit 772 and the adaptablesorter unit 776 can communicate with each other in various methods toachieve the desired inspection and sorting functions (the adaptableinspection unit 772 performing the functions of the inspection deviceand the adaptable sorter unit 776 performing the functions of a slavedevice).

FIG. 39 is a block diagram of an adaptable inspection unit. Theadaptable inspection unit 790 that includes an attachment mechanism 791,an inspection sensor device (optical receiver) 792, a data port 793, anda power port 794. The adaptable inspection unit 790 may also include amemory unit and a processor capable of controlling the inspection sensordevice and writing information transmitted via the data port.

FIG. 40 is a block diagram of an adaptable sorter unit. The adaptablesorter unit 800 that includes an attachment mechanism 801, a sortingdevice 802, a data port 803, and a power port 804. The adaptable sorterunit 800 may also include a memory and a processor capable of readinginformation received via the data port and controlling the sortingdevice.

FIG. 41 is a flowchart 900 illustrating the operations performed by anadaptable inspection unit. In step 901, an attachment mechanism isconnected to the adaptable inspection unit. In step 902, the attachmentmechanism is connected to the existing processing line. This can be aconnection directly to the existing processing line or to an object nearthe existing processing line, such as a wall, ceiling, mounting stand,or conveyor sidewall. In step 903, a power port of the adaptableinspection unit is connected to a power source. In step 904, a data portof the adaptable inspection unit is connected to a data communicationchannel. The data communication channel can be a wired or wirelesschannel. In step 905, the existing processing line is run with theadaptable inspection unit in place and executing. In step 906, theexisting processing line equipment is capable of performing automatedinspection.

FIG. 42 is a flowchart 910 illustrating the operations performed by anadaptable sorting unit. In step 911, an attachment mechanism isconnected to the adaptable sorter unit. In step 912, the attachmentmechanism is connected to the existing processing line. This can be aconnection directly to the existing processing line or to an object nearthe existing processing line, such as a wall, ceiling, mounting stand,or conveyor sidewall. In step 913, a power port of the adaptable sorterunit is connected to a power source. In step 914, a data port of theadaptable sorter unit is connected to a data communication channel. Thedata communication channel can be a wired or wireless channel. In step915, the existing processing line is run with the adaptable sorter unitin place and executing. In step 916, the existing processing lineequipment is capable of performing automated sorting.

Given the new methods and apparatuses disclosed above, an existingprocessing line can be quickly and inexpensively retrofitted to performautomated inspection and automated sorting, which results in (i)improved inspection quality and reliability, (ii) improved sortingaccuracy and reliability, (iii) improved throughput capability, and (iv)reduced operating costs.

The exemplary embodiments described above discuss adaptable inspectionunits and adaptable sorter units attached to a conveyor. However, oneskilled in the art will readily appreciate that the adaptable inspectionunits and adaptable sorter units may also be attached to any other typeof existing processing line, such as a chute in a similar manner toattain similar functionality and benefits.

Vacuum Adaptable Sorting Unit

As disclosed above, many different sorting methods may be implemented inan adaptable sorting unit. One of those sorting methods includesgenerating a vacuum that is applied to specific samples so to sort themout from a group of samples.

Vacuum sorting is desirable due to its high level of performance andcontrollability. However, the presence of a vacuum source in sortingfacilities is rare. Moreover, installation of vacuum systems in sortingfacilities is prohibitively expensive. Therefore, implementation of avacuum sorting system has been economically infeasible.

While vacuum sources are rare in sorting facilities, pressurized airsystems are often present in sorting facilities. If one we able tocreate a vacuum sorting unit utilizing a pressurized air system, thenimplementation of a vacuum sorting system would not be prohibitivelyexpensive. A solution for implementing a vacuum sorting unit using apressurized air system is disclosed herein.

An exemplary setup is illustrated in FIG. 43 . A conveyor 1,000 is usedto propel samples from left to right. An adaptable inspection unit 1,002is used to capture one or more images of the samples as they within thefield of view of the adaptable inspection unit 1,002. In one example,the adaptable inspection unit is an optical inspector. Each of theimages is processed to determine the location of each sample on theconveyor and whether or not each sample should be sorted. If thedecision is that a sample should be sorted, then the location and timingof the vacuum application to the sample is calculated. When the sampleis located within the vacuum suction area of adaptable sorting unit1,004, the vacuum is activated and the sample to be sorted 1,006 issucked into the adaptable sorting unit 1,004.

The inspection of the one or more images acquired by the adaptableinspection unit 1,002 are stored in a memory and processed by aprocessor. The location and timing of the vacuum application to thesample is calculated by the processor. In one example, the memory andprocessor are included in adaptable inspection unit 1,002. In anotherexample, the one or more images acquired by the adaptable inspectionunit 1,002 are communicated to a memory and processor located outside ofthe adaptable inspection unit 1,002. The adaptable inspection unit 1,002includes a communication modem. The communication modem can be anycommunication modem well known in the art. In one example, the modem isa wired technology such as Ethernet. In another example, the modem is awireless technology, such as WiFi or cellular (4G/5G) that utilizesantenna 1,003.

Control of the adaptable sorter unit 1,004 is performed, at least inpart, by a processor and a memory. In one example, the processor andmemory may be included in the adaptable sorter unit 1,004. In anotherexample, the processor and memory are located outside of the adaptablesorter unit 1,004. The adaptable sorter unit 1,004 includes acommunication modem. The communication modem can be any communicationmodem well known in the art. In one example, the modem is a wiredtechnology such as Ethernet. In another example, the modem is a wirelesstechnology, such as WiFi or cellular (4G/5G) that utilizes antenna1,005.

FIG. 44 is a more detailed diagram of the adaptable sorter unit 1,004.The adaptable sorter unit 1,004 includes a Venturi vacuum 1,001 and apneumatic valve 1,010. Optionally, the adaptable sorter unit 1,004 mayalso include a sorted sample container 1,012 and an optional antenna1,005 to allow wireless communication with the adaptable sorter unit.1,004. The Venturi vacuum 1,011 includes an inlet 1,007 and an outlet1,008. The pneumatic valve 1,010 includes a pressurized air inlet 1,009.

In operation, a pressurized air source is connected to the compressedair inlet 1,009 of pneumatic valve 1,010. The pneumatic valve 1,010 iscontrolled by an electrically controlled valve. When a first electriccontrol signal is present on the pneumatic valve input terminal, thevalve is closed. When a second electric control signal is present on thepneumatic valve input terminal, the valve is opened. In this fashion,the adaptable sorter unit 1,004 is engaged and disengaged.

In one example, the pneumatic valve includes a solenoid. The solenoidreceives the electrical control signal which causes the solenoid to moveinside the pneumatic valve so that the pneumatic air supplied throughthe compressed air inlet 1,010 can flow through the valve.

The output of the pneumatic valve 1,010 is connected to the pressurizedair input of the Venturi vacuum 1,011. The Venturi vacuum 1,011 therebycreates a vacuum force at inlet 1,007 and an outward force at outlet1,008. The operation of the Venturi vacuum 1,011 is illustrated in FIG.45 . It is noted that the illustration of FIG. 45 is only exemplary andis not required to implement the Venturi vacuum 1,011. Pressurize air isapplied to the nozzle 1,013 where air stream carries along ambient airin its turbulence and then passes through the mixer 1,014 on its wayout. This suction of ambient air creates a depression that generates thevacuum at inlet 1,007. The vacuum forces at inlet 1,007 can be setstrong enough to suck up a sample of desired weight and size through theinlet 1,007 and out of the outlet 1,008.

Once the sample is output through the outlet 1,008, the sample may landin a sorted sample container 1,012 (optional). In this fashion, all thesorted samples can be stored and routed to desired bins of samples.

It is noted that the sorted sample container 1,012 can also be a bucketor container that is positioned on the side of the conveyor. In thiscase, they may be a chute or flex hose going from the outlet 1,008 intothe bucket or container.

In another embodiment (not shown) a baseline vacuum control valve may beused in addition to the pneumatic valve disclosed above. Creating avacuum, with the Venturi system in combination with the pneumatic valve,may take a large amount of time to generate the vacuum force necessaryto sort a sample. In order to decrease that amount of time need togenerate the necessary vacuum force, a baseline vacuum control valve canbe added. The purpose of this baseline vacuum control valve is toprepare a baseline vacuum force that is not strong enough to move asample but decreases the time necessary to increase the vacuum force tothe level necessary to move a sample. In operation, as soon as theadaptable inspection unit captures an image of the sample, the baselinevacuum control valve will be actuated. As the baseline vacuum controlvalve will work on a lower incoming air pressure than the main pneumaticvalve, it will create a baseline vacuum force. Samples will not be movedby this baseline vacuum force because of the low suction force. Once thesample is within vacuum range, the main valve will open and the vacuumwill build up for its maximum level and thereby move the sample. In thisfashion, the baseline vacuum control valve is a preparation stage inwhich the vacuum is partially created (baseline vacuum force). The mainvalve is then only opened to quickly generate the full vacuum forcenecessary to move the sample.

In yet another embodiment, the pneumatic valve can be partially turnedon by adjusting the frequency of the electrical control signal. Inoperation, when a sample to be sorted is detected, the frequency of theelectrical control signal is sent to the pneumatic valve. The frequencyis set so that over time the average amount of pressurized air letthrough the valve is less than the maximum amount of pressurized airthat can pass through the valve when the valve is completely open over along time period. The valve will switch on when the electrical controlsignal is high and off when the electrical control signal is low. Ifthis occurs at fast enough rate, when seen over time, the effect is thesame as if the valve were fixed at a partially open position. This wouldcreate a baseline vacuum force with a low air flow that is not strongenough to move a sample into the adaptable sorter unit. When the sampleis within vacuum range, the electrical control signal changes towards asteady high (active) electrical control signal, thereby completelyopening the valve and creating a full vacuum in the Venturi systemnecessary to the sample into the adaptable sorting unit.

FIG. 46 is a front-view diagram of an adaptable inspection unit andvacuum adaptable sorter unit utilizing pressurized air with x-y-zlocation adjustment. The location of the adaptable sorter unit may beadjusted in all three dimensions (x,y,z).

The z-dimension may need to be adjusted to set the optimal height of theadaptable sorter unit so that the vacuum can be maximally applied topassing samples while ensuring that the sample do not come into directcontact with the adaptable sorter unit housing.

The x-y dimension can be adjusted so to properly position the adaptablesorter unit so it is able to apply a vacuum force to the desired sampleon the conveyor. In this fashion, a single adaptable sorter unit can beused to sort all samples on a single conveyer. In operation, theadaptable inspection unit acquires images used to determine the locationof all samples that require sorting, then the location information isused to position and trigger the adaptable sorter unit so that thedesired sample is sorted. In one example, the adaptable inspection unitis attached to a movable arm that receives a location control signal. Inone example, the location control signal includes coordinateinformation. Any automated movement technologies known in the art can beused to adjust the location of the adaptable inspection unit. Anylocation control technologies known in the art can be used tocommunicate the location information. The location informationcalculated by a processor, based at least in part, on an image capturedby the adaptable inspection unit. The processor then causes locationinformation to be sent to the adaptable sorter unit. The adaptablesorter unit then uses the location information to adjust the position ofthe adaptable sorter unit.

The triggering of the adaptable sorter unit is caused by a processor. Inone example, the processor calculates the time when the adaptable sorterunit is to be turned on, based at least in part on an image captured bythe adaptable inspection unit. In one example, the processor outputs atrigger signal that is communicated to the adaptable sorter unit. Inresponse to receiving the trigger signal, the adaptable sorter unitapplies the vacuum.

FIG. 47 is a top-down diagram of an adaptable inspection unit and vacuumadaptable sorter unit utilizing pressurized air with x-y-z locationadjustment. This diagram further illustrates the operation of theadaptable sorter unit with x-y-z location adjustment. The samples on theconveyor move in the direction 1,015. In one example, the adaptableinspection unit 1,002 has a viewable inspection area 1,016. As samplepass through the viewable inspection area 1,016, one or more images ofeach sample are captured. The one or more images of each sample are usedto determine if each sample is to be sorted. If a sample is to besorted, then the adaptable sorter unit 1,004 is moved to location alongthe path of the sample moving along the conveyor. Then, once the sampleis within the vacuum range of the adaptable sorter unit 1,004 theadaptable sorter unit 1,004 is activated by opening the pneumatic valvethereby causing the vacuum force and sucking the sample up from theconveyor and into the adaptable sorting unit 1,004. All unsorted samplescontinue along the conveyor.

FIG. 48 is a front-view diagram of an adaptable inspection unit andarray of fixed location vacuum adaptable sorter units utilizingpressurized air. The adaptable sorter unit array 1,017 includes nineadaptable sorter units (A-I). In one example, the number of adaptablesorter units is determined by the width of the conveyor and the width ofthe adaptable sorter unit so that the entire width of the conveyor iscovered by at least one adaptable sorter unit.

FIG. 49 is a top-down diagram of an adaptable inspection unit and arrayof fixed location vacuum adaptable sorter units utilizing pressurizedair. In operation, samples flow in direction 1,019 along the conveyor. Asample first passes through the viewable inspection area 1,020 where oneor more images of the sample are acquired. The one or more images arethen used to determine if the sample is to be sorted. If the sample isto be sorted, then the location of the sample is used to control whichadaptable sorter unit (A-I) of the adaptable sorter unit array 1,017 isto be used to sort the sample. The timing at which the selectedadaptable sorter unit is triggered to enable the vacuum force iscalculated based at least in part on the time and location when theimage of the sample was captured and the speed at which the sampletravels along the conveyor. The selected adaptable sorter unit is thentriggered at the calculated time so to sort the sample. All unsortedsamples continue along the conveyor.

FIG. 50 is a perspective diagram of an adaptable inspection unit andarray of fixed location vacuum adaptable sorter units utilizingpressurized air. As described regarding FIG. 49 , the sample moves alongthe conveyor. First, one or more images of the sample are captured bythe adaptable inspection unit 1,018. The one or more images of thesample are then used to select at least one of the adaptable sorterunits in the adaptable sorter unit array 1,017. The selected one or moreadaptable sorter units are then triggered (turned on) when the sample iswithin the vacuum force of the adaptable sorter unit.

FIG. 50 also illustrates an example where the sorted sample container ofeach adaptable sorter unit is connected together. This configurationallows all the sorted samples from each adaptable sorter unit to flow tothe same sample sorting bin. It is noted that in other examples one ormore adaptable sorter units may have individual sorted sample containersso that the sorted samples are not binned together.

It is noted that the sorted sample container can also be a bucket orcontainer that is positioned on the side of the conveyor. In this case,they may be a chute or flex hose going from the outlet into the bucketor container.

It is also noted, that while the above examples discuss a conveyor, thesame adaptable system can be implemented on any processing line, such asa chute.

FIG. 51 is a flowchart 1,030 describing the steps of enabling a vacuumadaptable sorter unit that utilizes pressurized air. In step 1,031, theattachment mechanism is connected to the adaptable sorter unit. In step1,032 the attachment mechanism is attached to the existing processingline. In step 1,033, a pressurized air line is connected to thepressurized air inlet. In step 1,034, pneumatic valve control isconnected to the pneumatic valve. In step 1,035, the existing processingline is run as normal. In step 1,036, the processing line equipment isnot able to perform sample sorting using vacuum sorting.

Sub Stream Auto Sampling

When processing a large number of samples, it may be desirable toinspect a portion of the total amount of samples being processed. Thisportion of the total samples being processed is referred to as a substream of samples. The total number of samples being processed isreferred to as the main stream of samples.

The inspected sub stream of samples can be inspected for many differentquality characteristics including, but not limited to: shape quality(based on matching a predetermined shape within a specified tolerance,then the sample is determined to be defective with respect to shape),surface contour quality (when a flat surface is greater than a specifiedarea, then the sample is determined to be defective with respect tosurface contour), hole quality (presence of holes in the sample), pestquality (presence of insects in/or on the sample), color quality(irregular color of the sample), size quality (irregular size of thesample), moisture level, oil content, fat content, and mycotoxincontent. In one example, a group of quality characteristics are referredto as inspection data 415. FIG. 18 , FIG. 19 , and FIG. 20 illustratevarious examples of inspection data.

FIG. 52 is a diagram that illustrates a sub stream inspection systemconfigured to inspect a sub stream of samples. The main stream ofsamples 1,100 are fed into the main stream chute 1,101. A portion ofmain stream of samples 1,100 fall into sub stream chute 1,102. The substream of samples 1,205 are then routed to inspection unit 1,102 by substream chute 1,102. Inspection unit 1,103 then performs inspection ofone or more of the sub stream of samples. In one example, the inspectorunit 1,103 is an in-flight 3D inspector described above herein. Afterthe sub stream of samples 1,105 are then routed back to the main streamof samples via inspection unit output chute 1,104. The sub stream ofsamples 1,105 and the main stream of samples 1,100 are then dropped ontoa conveyor 1,106. Conveyor 1,106 can be, but is not limited to, theconveyor types listed below.

-   -   Aero-mechanical conveyors    -   Automotive conveyors    -   Belt conveyor    -   Belt-driven live roller conveyors    -   Bucket conveyor    -   Chain conveyor    -   Chain-driven live roller conveyor    -   Drag conveyor    -   Dust-proof conveyors    -   Electric track vehicle systems    -   Flexible conveyors    -   Gravity conveyor    -   Gravity skate wheel conveyor    -   Lineshaft roller conveyor    -   Motorized-drive roller conveyor    -   Overhead I-beam conveyors    -   Overland conveyor    -   Pharmaceutical conveyors    -   Plastic belt conveyors    -   Pneumatic conveyors    -   Screw or auger conveyor    -   Spiral conveyors    -   Vertical conveyors    -   Vibrating conveyors    -   Wire mesh conveyors

FIG. 53 is a diagram of the system illustrated in FIG. 54 . The mainstream of samples 1,110 flows into a sorting unit 1,111. A sub stream ofsamples 1,112 are diverted by the sorting unit 1,111 toward inspectionunit 1,113. The remainder of the main stream of samples flow through thesorting unit 1,111 without being diverted toward the inspection unit1,113. Sorting unit 1,111 may be, but is not limited to, the followingtypes of sorting unit: a vacuum sorter, a mechanical pedal sorter, or anair jet sorter. The sub stream of samples 1,112 are then inspected byinspection 1,113. The sub stream of samples 1,112 are then routed backto the main stream of samples 1,110.

FIG. 54 is a flowchart 1,120 of the sub stream inspection systemillustrated in FIG. 53 . In step 1,121 a sub stream of samples from themain stream of samples is diverted toward an inspection unit. In step1,122, the diverted sub stream of samples is inspected. In step 1,123,the diverted sub stream of samples is routed back into the main streamof samples.

FIG. 55 is a diagram illustrating a sub stream inspection and weighingsystem. A main stream of samples 1,130 are sorted by sorting unit 1,131.Sorting unit 1,131 may be, but is not limited to, the following types ofsorting unit: a vacuum sorter, a mechanical pedal sorter, or an air jetsorter. The sorting unit 1,131 diverts a sub stream of samples 1,132toward inspection unit 1,133. The sub stream of samples is inspected byinspection unit 1,133. The sub stream of samples is then weighed byweighing unit 1,134. In one example, the weighing unit 1,134 is anelectronic scale that utilizes a strain gauge to measure weight. Inanother example, the weighing unit 1,134 is an electronically scale thatutilizes a force transducer. To aid in the routing of samples, theweighing unit may also include an output chute that directs samplesleaving the weighing unit 1,134. The weighing unit 1,134 can weigh eachsample of the sub stream of samples 1,132 individually or can weighmultiple samples from the sub stream of samples together and provide acalculated average weight per sample. The sub stream of samples is thencombined back into the main stream of samples.

It is noted that the data collected by both the inspection unit 1,133and the weighing unit 1,134 can be stored in various methods.

In one example, the data collected by the inspection unit 1,133 and theweighing unit 1,134 can be stored in a memory device included in theinspection unit 1,133. The memory device may be any type of memory knownin the art, such as, but not limited to, a disk drive, a solid statedrive, and a flash drive. The data collected by the weighing unit 1,134may be communicated to the inspection device 1,133 by a wire or wirelesscommunication protocol (RS-232, WiFi, Bluetooth, ZigBee, Ethernet, etc.)

In another example, the data collected by the inspection unit 1,133 andthe weighing unit 1,134 can be stored in a memory device included in theweighing unit 1,134. The memory device may be any type of memory knownin the art, such as, but not limited to, a disk drive, a solid statedrive, and a flash drive. The data collected by the inspecting unit1,133 may be communicated to the weighing unit 1,134 by a wire orwireless communication protocol (RS-232, WiFi, Bluetooth, ZigBee,Ethernet, etc.)

In yet another example, the data collected by the inspection unit 1,133and the weighing unit 1,134 can be stored in a memory device locatedoutside of the inspecting unit 1,133 and the weighing unit 1,134, suchas a server or remote networked computing device. The memory device maybe any type of memory known in the art, such as, but not limited to, adisk drive, a solid state drive, and a flash drive. The data collectedby a wire or wireless communication protocol (RS-232, WiFi, Bluetooth,ZigBee, Ethernet, etc.)

FIG. 56 is a flowchart 1,140 of a sub stream inspection and weighingsystem. In step 1,141 a sub stream of samples from the main stream ofsamples is diverted toward an inspection unit. In step 1,142, thediverted sub stream of samples is inspected. In step 1,143, the substream of samples is routed to a weighing unit. In one example, theweighing unit 1,134 is an electronic scale that utilizes a strain gaugeto measure weight. In another example, the weighing unit 1,134 is anelectronically scale that utilizes a force transducer. To aid in therouting of samples, the weighing unit may also include an output chutethat directs samples leaving the weighing unit 1,1342. In step 1,144,the sub stream of samples is weighed. Each sample of the sub stream ofsamples can be weighed individually or can be weighed with multiplesamples from the sub stream of samples together and provide a calculatedaverage weight per sample. In step 1,145, the sub stream of samples isrouted to the main stream of samples.

FIG. 57 is a diagram illustrating a sub stream weighing and inspectionsystem. A main stream of samples 1,150 are sorted by sorting unit 1,151.Sorting unit 1,151 may be, but is not limited to, the following types ofsorting unit: a vacuum sorter, a mechanical pedal sorter, or an air jetsorter. The sorting unit 1,151 diverts a sub stream of samples 1,152toward weighing unit 1,153. The sub stream of samples is weighed byweighing unit 1,153. In one example, the weighing unit 1,153 is anelectronic scale that utilizes a strain gauge to measure weight. Inanother example, the weighing unit 1,153 is an electronically scale thatutilizes a force transducer. To aid in the routing of samples, theweighing unit may also include an output chute that directs samplesleaving the weighing unit 1,153. The weighing unit 1,153 can weight eachsample of the sub stream of samples 1,152 individually or can weighmultiple samples from the sub stream of samples together and provide acalculated average weight per sample. The sub stream of samples is theninspected by inspecting unit 1,154. The sub stream of samples is thencombined back into the main stream of samples.

It is noted that the data collected by both the inspection unit 1,153and the weighing unit 1,154 can be stored in various methods.

In one example, the data collected by the inspection unit 1,154 and theweighing unit 1,153 can be stored in a memory device included in theinspection unit 1,154. The memory device may be any type of memory knownin the art, such as, but not limited to, a disk drive, a solid statedrive, and a flash drive. The data collected by the weighing unit 1,153may be communicated back to the inspection unit 1,154 by a wire orwireless communication protocol (RS-232, WiFi, Bluetooth, ZigBee,Ethernet, etc.)

In another example, the data collected by the inspection unit 1,154 andthe weighing unit 1,153 can be stored in a memory device included in theweighing unit 1,153. The memory device may be any type of memory knownin the art, such as, but not limited to, a disk drive, a solid statedrive, and a flash drive. The data collected by the inspecting unit1,154 may be communicated back to the weighing unit 1,153 by a wire orwireless communication protocol (RS-232, WiFi, Bluetooth, ZigBee,Ethernet, etc.)

In yet another example, the data collected by the inspection unit 1,154and the weighing unit 1,153 can be stored in a memory device locatedoutside of the inspecting unit 1,154 and the weighing unit 1,153, suchas a server or remote networked computing device. The memory device maybe any type of memory known in the art, such as, but not limited to, adisk drive, a solid state drive, and a flash drive. The data collectedby a wire or wireless communication protocol (RS-232, WiFi, Bluetooth,ZigBee, Ethernet, etc.)

FIG. 58 is a flowchart 1,160 of a sub stream weighing and inspectionsystem. In step 1,161 a sub stream of samples from the main stream ofsamples is diverted toward a weighing unit. In step 1,162, the divertedsub stream of samples is weighed. Each sample of the sub stream ofsamples can be weighed individually or can be weighed with multiplesamples from the sub stream of samples together and provide a calculatedaverage weight per sample. In step 1,163, the sub stream of samples isrouted to an inspection unit. In step 1,164, the sub stream of samplesis inspected. In step 1,165, the sub stream of samples is routed to themain stream of samples.

FIG. 59 is a diagram illustrating a sub stream inspection and collectionsystem. A main stream of samples 1,170 are sorted by sorting unit 1,171.Sorting unit 1,171 may be, but is not limited to, the following types ofsorting unit: a vacuum sorter, a mechanical pedal sorter, or an air jetsorter. The sorting unit 1,171 diverts a sub stream of samples 1,172toward inspection unit 1,173. The sub stream of samples is inspected byinspection unit 1,173. The sub stream of samples is then routed to acollection unit 1,175. The collection unit 1,175 serves to store the substream of samples after the inspection is completed. Collecting theinspected sub stream of samples, instead of recombing the sub stream ofsamples with the main stream of samples, is advantageous in that itallows for later examination of the sub stream of samples that wereinspected. For example, it may be advantageous to manually inspect thesub stream of samples and compare the manual inspection results with theinspection data provide by the inspection unit 1,173.

It is noted that the data collected by inspection unit 1,173 can bestored in various methods.

In one example, the data collected by the inspection unit 1,174 can bestored in a memory device included in the inspection unit 1,174. Thememory device may be any type of memory known in the art, such as, butnot limited to, a disk drive, a solid state drive, and a flash drive.

In yet another example, the data collected by the inspection unit 1,174can be stored in a memory device located outside of the inspecting unit1,174 such as a server or remote networked computing device. The memorydevice may be any type of memory known in the art, such as, but notlimited to, a disk drive, a solid state drive, and a flash drive. Thedata collected by a wire or wireless communication protocol (RS-232,WiFi, Bluetooth, ZigBee, Ethernet, etc.)

FIG. 60 is a flowchart 1,180 of a sub stream inspection and collectionsystem. In step 1,181 a sub stream of samples from the main stream ofsamples is diverted toward an inspection unit. In step 1,182, thediverted sub stream of samples is inspected. In step 1,183, the substream of samples is routed to a collection unit.

FIG. 61 is a diagram illustrating a sub stream inspection, weighing andcollection system. A main stream of samples 1,190 are sorted by sortingunit 1,191. Sorting unit 1,191 may be, but is not limited to, thefollowing types of sorting unit: a vacuum sorter, a mechanical pedalsorter, or an air jet sorter. The sorting unit 1,191 diverts a substream of samples 1,192 toward inspection unit 1,193. The sub stream ofsamples is inspected by inspection unit 1,193. The sub stream of samplesis then weighed by weighing unit 1,194. In one example, the weighingunit 1,194 is an electronic scale that utilizes a strain gauge tomeasure weight. In another example, the weighing unit 1,194 is anelectronically scale that utilizes a force transducer. To aid in therouting of samples, the weighing unit may also include an output chutethat directs samples leaving the weighing unit 1,194. The weighing unit1,194 can weigh each sample of the sub stream of samples 1,192individually or can weigh multiple samples from the sub stream ofsamples together and provide a calculated average weight per sample. Thesub stream of samples is then routed to collection unit 1,196. Thecollection unit 1,196 serves to store the sub stream of samples afterthe inspection is completed. Collecting the inspected sub stream ofsamples, instead of recombing the sub stream of samples with the mainstream of samples, is advantageous in that it allows for laterexamination of the sub stream of samples that were inspected. Forexample, it may be advantageous to manually inspect the sub stream ofsamples and compare the manual inspection results with the inspectiondata provide by the inspection unit 1,193 and weighing unit 1,194.

It is noted that the data collected by both the inspection unit 1,193and the weighing unit 1,194 can be stored in various methods.

In one example, the data collected by the inspection unit 1,193 and theweighing unit 1,194 can be stored in a memory device included in theinspection unit 1,193. The memory device may be any type of memory knownin the art, such as, but not limited to, a disk drive, a solid statedrive, and a flash drive. The data collected by the weighing unit 1,194may be communicated to the inspection device 1,193 by a wire or wirelesscommunication protocol (RS-232, WiFi, Bluetooth, ZigBee, Ethernet, etc.)

In another example, the data collected by the inspection unit 1,193 andthe weighing unit 1,194 can be stored in a memory device included in theweighing unit 1,194. The memory device may be any type of memory knownin the art, such as, but not limited to, a disk drive, a solid statedrive, and a flash drive. The data collected by the inspecting unit1,193 may be communicated to the weighing unit 1,194 by a wire orwireless communication protocol (RS-232, WiFi, Bluetooth, ZigBee,Ethernet, etc.)

In yet another example, the data collected by the inspection unit 1,193and the weighing unit 1,194 can be stored in a memory device locatedoutside of the inspecting unit 1,193 and the weighing unit 1,194, suchas a server or remote networked computing device. The memory device maybe any type of memory known in the art, such as, but not limited to, adisk drive, a solid state drive, and a flash drive. The data collectedby a wire or wireless communication protocol (RS-232, WiFi, Bluetooth,ZigBee, Ethernet, etc.)

FIG. 62 is a flowchart 1,200 of a sub stream inspection, weighing andcollection system. In step 1,201 a sub stream of samples from the mainstream of samples is diverted toward an inspection unit. In step 1,202,the diverted sub stream of samples is inspected. In step 1,203, the substream of samples is routed to a weighing unit. In step 1,204, the substream of samples is weighed. Each sample of the sub stream of samplescan be weighed individually or can be weighed with multiple samples fromthe sub stream of samples together and provide a calculated averageweight per sample. In step 1,205, the sub stream of samples is routed toa collection unit.

FIG. 63 is a diagram illustrating a sub stream weighing, inspection, andcollection system. A main stream of samples 1,210 are sorted by sortingunit 1,211. Sorting unit 1,211 may be, but is not limited to, thefollowing types of sorting unit: a vacuum sorter, a mechanical pedalsorter, or an air jet sorter. The sorting unit 1,211 diverts a substream of samples 1,212 toward weighing unit 1,213. In one example, theweighing unit 1,213 is an electronic scale that utilizes a strain gaugeto measure weight. In another example, the weighing unit 1,213 is anelectronically scale that utilizes a force transducer. To aid in therouting of samples, the weighing unit may also include an output chutethat directs samples leaving the weighing unit 1,213. The weighing unit1,213 can weigh each sample of the sub stream of samples 1,212individually or can weigh multiple samples from the sub stream ofsamples together and provide a calculated average weight per sample. Thesub stream of samples is weighed by weighing unit 1,213. The sub streamof samples is then inspected by inspecting unit 1,214. The sub stream ofsamples is then routed to collection unit 1,216. The collection unit1,216 serves to store the sub stream of samples after the inspection iscompleted. Collecting the inspected sub stream of samples, instead ofrecombing the sub stream of samples with the main stream of samples, isadvantageous in that it allows for later examination of the sub streamof samples that were inspected. For example, it may be advantageous tomanually inspect the sub stream of samples and compare the manualinspection results with the inspection data provide by the inspectionunit 1,214 and weighing unit 1,213.

It is noted that the data collected by both the inspection unit 1,214and the weighing unit 1,213 can be stored in various methods.

In one example, the data collected by the inspection unit 1,214 and theweighing unit 1,213 can be stored in a memory device included in theinspection unit 1,214. The memory device may be any type of memory knownin the art, such as, but not limited to, a disk drive, a solid statedrive, and a flash drive. The data collected by the weighing unit 1,213may be communicated to the inspection device 1,214 by a wire or wirelesscommunication protocol (RS-232, WiFi, Bluetooth, ZigBee, Ethernet, etc.)

In another example, the data collected by the inspection unit 1,214 andthe weighing unit 1,213 can be stored in a memory device included in theweighing unit 1,213. The memory device may be any type of memory knownin the art, such as, but not limited to, a disk drive, a solid statedrive, and a flash drive. The data collected by the inspecting unit1,214 may be communicated to the weighing unit 1,213 by a wire orwireless communication protocol (RS-232, WiFi, Bluetooth, ZigBee,Ethernet, etc.)

In yet another example, the data collected by the inspection unit 1,214and the weighing unit 1,213 can be stored in a memory device locatedoutside of the inspecting unit 1,214 and the weighing unit 1,213, suchas a server or remote networked computing device. The memory device maybe any type of memory known in the art, such as, but not limited to, adisk drive, a solid state drive, and a flash drive. The data collectedby a wire or wireless communication protocol (RS-232, WiFi, Bluetooth,ZigBee, Ethernet, etc.)

FIG. 64 is a flowchart 1,220 of a sub stream weighing, inspection, andcollection system. In step 1,221 a sub stream of samples from the mainstream of samples is diverted toward a weighing unit. In step 1,222, thediverted sub stream of samples is weighed. Each sample of the sub streamof samples can be weighed individually or can be weighed with multiplesamples from the sub stream of samples together and provide a calculatedaverage weight per sample. In step 1,223, the sub stream of samples isrouted to an inspection unit. In step 1,224, the sub stream of samplesis inspected. In step 1,225, the sub stream of samples is routed to acollection unit.

Product Target Quality Control System with Intelligent Source Control

In most industries customers only want to pay a fee that is commensuratewith the quality level necessary to successfully run their business.While businesses would prefer to always receive the highest gradeproduct, the reality is that higher grade products come with anincreased price that may not be practical or necessary. Similarly,product producers may have a required product quality they are requiredto provide to a specific customer. Even though the producer of theproduct knows that their Grade “A” is above and beyond the requiredquality level, if their Grade “B” does not meet the required qualitylevel, the product producer is forced to sell product of a higher gradethan required by the customer's specifications, thereby losing potentialprofits that could have been attained by selling the higher thanrequired quality product for a higher price. The invention describedherein provides various solutions to this problem.

FIG. 65 is a flowchart diagram of a target quality control system. Inone example, grade “A” product 1,250 contains product pieces, orsamples, that are of a relatively high quality level, grade “B” product1,251 contains product pieces, or samples, that are of a relativelymedium quality level, and grade “C” 1,253 contains product pieces, orsamples, that are of a relatively low quality level. The approximaterange of quality for each grade is known. Given this setup, the systemcan output product at Grade “A”, Grade “B” or Grade “C”, however, asmentioned above, it may be desirable to output a more specific productquality grade. In this scenario, there is a need to provide a morespecific desired quality level than is provided in the presorted productgroups.

In one embodiment of the present invention, this goal is achieved by (i)providing a means for variable selecting product from each of theproduct grade groups, (ii) providing a means for inspecting the productselected from each product grade group, and (iii) providing a feedbackmeans, wherein the measured quality values generated based on theproduct inspection are utilized to adjust the ratio of product selectedfrom each of the product grade groups. The product inspector 1,255provides the means for inspecting the selected product. The computingdevice 1,256 provides the means for controlling the product ratio basedon the measured quality values. Selector 1,254 provides the means tovariable select product form each product grade group. Specific desiredquality of product 1,257 provides the means to collect or direct thespecific mix of product with the desired specific quality.

It is noted herein that FIG. 65 is only a high-level system diagram andthat various parts of the system may be arranged differently to attainthe same solution. The different arrangements are clear to one skilledin the art after reviewing this description. Specific variations of thishigh-level system are provided below.

FIG. 66 is an operational diagram of a first target quality controlsystem. The system includes a first sample source 1,301, a second samplesource 1,302 and up to an Nth sample source 1,303. Each source containsproduct of a different predetermined quality level. In one example, thefirst sample source contains product of a quality grade “A”, the secondsample source contains product of quality grade “B, and the Nth samplesource contains product of quality grade “N”. In this example, qualitygrade “A” is greater than quality grade “B” and quality grade “B” isgreater than quality grade “N”. In one example, each sample source maybe a product bin filled with product of a known quality level. The binmay be gravity fed and output product via an output chute.

Source selecting device(s) 1,304 includes a separate input that iscoupled to the output of each sample source. In this manner, the sourceselecting device 1,304 is able to receive product (sample) from eachsample source. Source selecting device 1,304 includes the means to passproduct (sample) from any desired product input to a single productoutput port. The product input port is controlled by command 1,313.Command 1,313 can be received by either a wired or wirelesscommunication. In the wired example, the source selecting device 1,304includes a wired communication port, such as but not limited to,Ethernet, RS-232, RS-485, USB, SCADA, etc. In the wireless example, thesource selecting device 1,304 includes a wireless communication port,such as but not limited to, WiFi, Bluetooth, ZigBee, cellular, etc. Theproduct output port of source selecting device 1,304 is coupled to theinput port of inspection device(s) 1,305. Selecting device 1,304 may beany sorter known in the art or any sorter described above in the presentapplication, such as but not limited to, the use of a burst of air toredirect the trajectory of a sample as it travels along the processingline, a vacuum that causes the sample to be removed from the flow ofsamples through the processing line, or the use of a mechanicallycontrolled flap to redirect the sample as it travels along theprocessing line.

The inspection devices 1,305 inspect the product (samples) output bysource selecting device 1,304. The inspection devices 1,305 have themeans to inspect the product (sample) output the source selecting device1,304 and means to generate measured quality value(s) of the product(sample). In one example, the inspection means is a system of one ormore cameras that capture one or more images of the product (sample). Inanother example, the means to generate measured quality value(s) isperformed by conducting image analysis to determine characteristics ofthe product (sample). In one example, the measured quality value(s)include image(s) 1,308 and/or inspection data 1,309. The measuredquality value(s) may be communicated via wired or wirelesscommunication, as mentioned above. The product is then output from theinspection device(s) 1,305 along the direction of sample flow 1,306 tocollector 1,307. In one embodiment, the inspection device 1,305 may beany of the inspectors described above, such as, an in-flight 3Dinspector, a 2 dimensional inspectors, or any inspection device that iscapable of detecting characteristics of a sample.

Collector 1,307 (optional) serves to collect all samples that passthrough the system. In one example, the collector 1,307 is a stainlesssteel bin. It is noted herein, that collector 1,307 is an optionalelement and not necessary. In other embodiments, the samples arealternatively routed to the remaining processing line.

The target quality control system further includes a computing device1,310. Computing device 1,310 receives the measured quality value(s) andoutputs the command 1,313 sent to the source selecting device 1,304. Thecomputing device 1,310 also receives a target quality value 1,312 andsource quality values(s) 1,311.

In operation, the computing device 1,310 controls the product qualitylevel of each product (sample) that is routed to collector 1,307. Eachselected product passes through the source selecting device 1,304 andinspection device(s) 1,305 before arriving at collector 1,307.Inspection device(s) 1,305 generate measured quality value(s) of theproduct (sample) and communicate the measured quality value(s) tocomputing device 1,310. Accordingly, computing device 1,310 utilizes themeasured quality value(s), the source quality value(s) 1,311 and thetarget quality value 1,312 to determine command 1,313 output bycomputing device 1,310. For example, computing device 1,310 may receivemeasured quality value(s) for a plurality of products (samples) thathave been routed to collector 1,307. In response to analyzing all of thereceived measure quality values, computing device 1,310 may determinethat average quality level of the products (samples) routed to collector1,307 is below a desired quality level indicated by target quality value1.312. In response to this determination, computing device 1,310 furtherdetermines that the next product (sample) passed to collector 1,307should of grade “A” (highest quality) so to attempt to increase theaverage quality of the products (samples) routed to collector 1,307.Accordingly, computing device 1,310 outputs a command 1,313 thatindicates that a product (sample) should be routed from sample source1,301 (containing grade “A” product) to collector 1,307. As the grade“A” product (sample) passes through inspection device(s) 1,305 a newmeasured quality value(s) is generated and communicated to computingdevice 1,310. Computing device 1,310 can then determine if the averageproduct quality level of all products (samples) in collector 1,307 areabove the desired quality level indicated by target quality value 1,312.

In another example, the command 1,310 may cause product (samples) to bepassed into the system from multiple sample sources contemporaneously.For example, if a target quality value is between grade “A” and grade“B” quality levels, then the command 1,310 may be set to output an evenamount form both sample source 1,301 and sample source 1,302, therebyinputting a blend of the two sources into the system.

It is noted herein that computing device 1,310 can use the targetquality values in various ways to control the output product quality. Inone example, the target quality value 1,312 can be used a thresholdvalue as described above. In another example, the target value 1,312 canbe used to set an acceptable range of quality values for the outputgroup of products (samples).

It is further noted herein that the target quality value 1,312 may varyover time as requirements change. For example, a customer qualityrequirement may increase or decrease due to customer demands. Thepresent invention allows for rapid on demand updating of the targetquality value 1,312, which in turn will allow the system to immediatelychange the product (sample) selection to bring the average quality ofall products (samples) in collector 1,307 to the desired level. It isnoted herein, that collector 1,307 is an optional element and notnecessary. In other embodiments, the samples are alternatively routed tothe remaining processing line.

In one example, the sample sources, the source selecting device(s), theinspection device(s) and the collector are all located at the samefacility so that the product (sample) can travel to each of these partsof the system. However, computing device 1,310 may or may not be locatedat the same facility. In one example, the computing device 1,310 may belocated in close proximity to inspection device(s) 1,305 and connectedvia an Ethernet connection. In another example, the computing device1,310 may be located on the other side of the world from the inspectiondevice(s) 1,305 and connected via a group of various communicationtechnologies, both wired and wireless.

Another note is made with respect to the inspection data 1,309 output byinspection device(s) 1,305. In one example, the inspection device(s) mayonly output captured raw inspection data, such as but not limited to,image(s) 1,308. The captured raw data is then analyzed by computingdevice 1,310 to determine the quality of the product (sample) inspected.In another example, the inspection device(s) may also analyze thecaptured raw data, such as but not limited to, image(s) 1,308 andgenerate inspection data 1,309, which is output to computing device(s)1,310. In this fashion, the present invention may utilize eitheradvanced smart inspection devices or simple inspection devices that onlycapture raw data.

It is also noted herein that computing device 1,310 may be implementedby one or a plurality of computing devices. In one example, computingdevice 1,310 may be a single computer. In another example, computingdevice 1,310 may be an array of computers or servers that performvarious computations of the system.

One or more source selecting device(s) may be utilized in the presentinvention. In one example, a single source selecting device has an inputport connected to each sample source and a single output port. Inanother example, a plurality of source selecting devices have inputports connected to various sample sources, and each source selectingdevice has an output port that is coupled to the product (sample) flowthat passes through the inspection device(s) 1,305 and collector 1,307.In this example, command 1,313 is communicated to each source selectingdevice and causes only one product (sample) to be output from one samplesource.

One or more inspection device(s) may be utilized in the presentinvention. In one example, all products (samples) are passed through asingle inspection device, which in turn outputs measured qualityvalue(s) for each product (sample). In another example, a plurality ofinspection device(s) may be utilized to inspect all products (samples).This second configuration may be utilized to increase the rate at whichproduct can be processed. The use of a plurality of inspection devices,allows for the inspection of multiple products (samples) at the samemoment in time. In this example, measured quality value(s) arecommunicated from each inspection device to computing device 1,310. Inone embodiment, the inspection device is an in-flight 3D inspector asdescribed above herein.

FIG. 67 is an operational diagram of a second target quality controlsystem. The system includes a first sample source 1,321, a second samplesource 1,322 and up to an Nth sample source 1,323. Each source containsproduct of a different predetermined quality level. In one example, thefirst sample source contains product of a quality grade “A”, the secondsample source contains product of quality grade “B, and the Nth samplesource contains product of quality grade “N”. In this example, qualitygrade “A” is greater than quality grade “B” and quality grade “B” isgreater quality grade “N”. In one example, each sample source may be aproduct bin filled with product of the known quality level. The bin maybe gravity fed and output product via an output chute.

The system does not include a source selecting device, but rather eachsample source includes a means to control the output of product(samples) from the sample source. In one example, the output controlmeans is a vibration plate that vibrates to cause a sample to exit fromthe sample source. In another example, the output control means is agate that is opened to cause a sample to exit from the sample source.The sample sources receive a source control instruction via command1,332 and cause a sample to output when instructed to do so. The outputof each sample source is coupled to inspection device(s) 1,324.

The inspection devices 1,324 inspect the product (samples) output by thesample sources. The inspection devices 1,324 have the means to inspectthe product (sample) output the sample sources and means to generatemeasured quality value(s) of the product (sample). In one example, theinspection means is a system of one or more cameras that capture one ormore images of the product (sample). In another example, the means togenerate measured quality value(s) is performed by conducting imageanalysis to determine characteristics of the product (sample). In oneexample, the measured quality value(s) include image(s) 1,327 and/orinspection data 1,328. The measured quality value(s) may be communicatedvia wired or wireless communication, as mentioned above. The product isthen output from the inspection device(s) 1,324 along the direction ofsample flow 1,325 to collector 1,326.

Collector 1,326 (optional) serves to collect all samples that passthrough the system. In one example, the collector 1,326 is a stainlesssteel bin. It is noted herein that collector 1,326 is an optionalelement and not necessary. In other embodiments, the samples arealternatively routed to the remaining processing line.

The target quality control system further includes a computing device1,329. Computing device 1,329 receives the measured quality value(s) andoutputs the command 1,332 sent to the sample sources. The computingdevice 1,329 also receives a target quality value 1,331 and sourcequality values(s) 1,330. The source quality value(s) 1,330 and targetquality value 1,331 can be received from a human interface device, suchas a computer terminal or touch screen device, or from another computersystem.

In operation, the computing device 1,329 controls the product qualitylevel of each product (sample) that is routed to collector 1,326. Eachselected product passes through the inspection device(s) 1,324 beforearriving at collector 1,326. Inspection device(s) 1,324 generatemeasured quality value(s) of the product (sample) and communicate themeasured quality value(s) to computing device 1,329. Accordingly,computing device 1,329 utilizes the measured quality value(s), thesource quality value(s) 1,330 and the target quality value 1,331 todetermine command 1,332 output by computing device 1,329. For example,computing device 1,329 may receive measured quality value(s) for aplurality of products (samples) that have been routed to collector1,326. In response to analyzing all of the received measure qualityvalues, computing device 1,329 may determine that average quality levelof the products (samples) routed to collector 1,326 is below a desiredquality level indicated by target quality value 1.331. In response tothis determination, computing device 1,329 further determines that thenext product (sample) passed to collector 1,326 should of grade “A”(highest quality) so to attempt to increase the average quality of theproducts (samples) routed to collector 1,326. Accordingly, computingdevice 1,329 outputs a command 1,332 that indicates that a product(sample) should be routed from sample source 1,321 (containing grade “A”product) to collector 1,326. As the grade “A” product (sample) passesthrough inspection device(s) 1,324 a new measured quality value(s) isgenerated and communicated to computing device 1,329. Computing device1,329 can then determine if the average product quality level of allproducts (samples) in collector 1,326 are above the desired qualitylevel indicated by target quality value 1,331.

In another example, the command 1,332 may cause product (samples) to bepassed into the system from multiple sample sources contemporaneously.For example, if a target quality value is between grade “A” and grade“B” quality levels, then the command 1,332 may be set to output an evenamount form both sample source 1,321 and sample source 1,322, therebyinputting a blend of the two sources into the system.

It is noted herein that computing device 1,329 can use the targetquality values in various ways to control the output product quality. Inone example, the target quality value 1,331 can be used a thresholdvalue as described above. In another example, the target value 1,331 canbe used to set an acceptable range of quality values for the outputgroup of products (samples).

It is further noted herein that the target quality value 1,331 may varyover time as requirements change. For example, a customer qualityrequirement may increase or decrease due to customer demands. Thepresent invention allows for rapid on demand updating of the targetquality value 1,331, which in turn will allow the system to immediatelychange the product (sample) selection to bring the average quality ofall products (samples) in collector 1,326 to the desired level. It isnoted herein that collector 1,326 is an optional element and notnecessary. In other embodiments, the samples are alternatively routed tothe remaining processing line.

In one example, the sample sources, the source selecting device(s), theinspection device(s) and the collector are all located at the samefacility so that the product (sample) can travel to each of these partsof the system. However, computing device 1,329 may or may not be locatedat the same facility. In one example, the computing device 1,329 may belocated in close proximity to inspection device(s) 1,324 and connectedvia an Ethernet connection. In another example, the computing device1,329 may be located on the other side of the world from the inspectiondevice(s) 1,324 and connected via a group of various communicationtechnologies, both wired and wireless.

Another note is made with respect to the inspection data 1,328 output byinspection device(s) 1,324. In one example, the inspection device(s) mayonly output captured raw inspection data, such as but not limited to,image(s) 1,327. The captured raw data is then analyzed by computingdevice 1,329 to determine the quality of the product (sample) inspected.In another example, the inspection device(s) may also analyze thecaptured raw data, such as but not limited to, image(s) 1,327 andgenerate inspection data 1,328, which is output to computing device(s)1,329. In this fashion, the present invention may utilize eitheradvanced smart inspection devices or simple inspection devices that onlycapture raw data.

It is also noted herein that computing device 1,329 may be implementedby one or a plurality of computing devices. In one example, computingdevice 1,329 may be a single computer. In another example, computingdevice 1,329 may be implemented utilizing an array of computers orservers that perform various computations of the system.

One or more inspection device(s) may be utilized in the presentinvention. In one example, all products (samples) are passed through asingle inspection device, which in turn outputs measured qualityvalue(s) for each product (sample). In another example, a plurality ofinspection device(s) may be utilized to inspect all products (samples).This second configuration may be utilized to increase the rate at whichproduct can be processed. The use of a plurality of inspection devices,allows for the inspection of multiple products (samples) at the samemoment in time. In this example, measured quality value(s) arecommunicated from each inspection device to computing device 1,329.

FIG. 68 is an operational diagram of a third target quality controlsystem. The system includes a first sample source 1,341, a second samplesource 1,342 and up to an Nth sample source 1,343. Each source maycontain product of a different or similar predetermined quality level.In one example, the first sample source contains product of a qualitygrade “A”, the second sample source contains product of quality grade“B, and the Nth sample source contains product of quality grade “N”. Inthis example, quality grade “A” is greater than quality grade “B” andquality grade “B” is greater quality grade “N”. In one example, eachsample source may be a product bin filled with product of the knownquality level. The bin may be gravity fed and output product via anoutput chute.

The system does not include a source selecting device, but rather eachsample source includes a means to control the output of product(samples) from the sample source. In one example, the output controlmeans is a vibration plate that vibrates to cause a sample to exit fromthe sample source. In another example, the output control means is agate that is opened to cause a sample to exit from the sample source.The sample sources receive a source control instruction via command1,354 and cause a sample to output when instructed to do so. The outputof each sample source is coupled to a dedicated inspection device thatreceives output product (samples) only from one sample source. Theone-to-one relationship between sample sources and inspection devicesprovide multiple benefits.

First, the one-to-one relationship between sample source and inspectiondevice allows the system to pinpoint the actual quality level of eachsample source. This system benefit is extremely useful, in that, thesource quality values are often human generated and based on a varietyof assumptions and estimates. The system described in the presentinvention allows for the exact measurement of the quality level and typeof defects present for each sample source. For example, a sample sourcemay be designated grade “A”, however, after inspection of large quantityof product (samples) from the source, the system may determine that thegrade of the sample source is in fact not grade “A”, but rather a lowergrade of product. This information is extremely important when trying tocontrol the exactly quality level of the output product.

Second, the one-to-one relationship between sample source and inspectiondevice provides increased system throughput. Inspecting multiple samplesat the same moment in time allows for more product (samples) to movethrough the system over a fixed period of time.

The inspection devices inspect the product (samples) output by thesample sources. The inspection devices have the means to inspect theproduct (sample) output the sample sources and means to generatemeasured quality value(s) of the product (sample). In one example, theinspection means is a system of one or more cameras that capture one ormore images of the product (sample). In another example, the means togenerate measured quality value(s) is performed by conducting imageanalysis to determine characteristics of the product (sample). In oneexample, the measured quality value(s) include image(s) 1,349 and/orinspection data 1,350. The measured quality value(s) may be communicatedvia wired or wireless communication, as mentioned above. The product isthen output from the inspection device(s) along the direction of sampleflow 1,347 to collector 1,348.

Collector 1,348 serves to collect all samples that pass through thesystem. In one example, the collector 1,348 is a stainless steel bin. Itis noted herein that collector 1,348 is an optional element and notnecessary. In other embodiments, the samples are alternatively routed tothe remaining processing line.

The target quality control system further includes a computing device1,341. Computing device 1,351 receives the measured quality value(s) andoutputs the command 1,354 sent to the sample sources. The computingdevice 1,351 also receives a target quality value 1,353 and sourcequality values(s) 1,352. The source quality value(s) 1,352 and targetquality value 1,353 can be received from a human interface device, suchas a computer terminal or touch screen device, or from another computersystem.

In operation, the computing device 1,351 controls the product qualitylevel of each product (sample) that is routed to collector 1,348. Eachselected product passes through the inspection devices 1,344, 1,345,1,346 before arriving at collector 1,348. Inspection devices generatemeasured quality value(s) of the product (sample) and communicate themeasured quality value(s) to computing device 1,351. Accordingly,computing device 1,351 utilizes the measured quality value(s), thesource quality value(s) 1,352 and the target quality value 1,353 todetermine command 1,354 output by computing device 1,351. For example,computing device 1,351 may receive measured quality value(s) for aplurality of products (samples) that have been routed to collector1,348. In response to analyzing all of the received measure qualityvalues, computing device 1,351 may determine that average quality levelof the products (samples) routed to collector 1,348 is below a desiredquality level indicated by target quality value 1.353. In response tothis determination, computing device 1,351 further determines that thenext product (sample) passed to collector 1,348 should be of grade “A”(highest quality) so to attempt to increase the average quality of theproducts (samples) routed to collector 1,348. Accordingly, computingdevice 1,351 outputs a command 1,354 that indicates that a product(sample) should be routed from sample source 1,341 (containing grade “A”product) to collector 1,348. As the grade “A” product (sample) passesthrough inspection device 1,344 a new measured quality value(s) isgenerated and communicated to computing device 1,351. Computing device1,351 can then determine if the average product quality level of allproducts (samples) in collector 1,348 is above the desired quality levelindicated by target quality value 1,353.

In another example, the command 1,354 may cause product (samples) to bepassed into the system from multiple sample sources contemporaneously.For example, if a target quality value is between grade “A” and grade“B” quality levels, then the command 1,354 may be set to output an evenamount form both sample source 1,341 and sample source 1,342, therebyinputting a blend of the two sources into the system.

It is noted herein that computing device 1,351 can use the targetquality values in various ways to control the output product quality. Inone example, the target quality value 1,353 can be used a thresholdvalue as described above. In another example, the target value 1,353 canbe used to set an acceptable range of quality values for the outputgroup of products (samples).

It is further noted herein that the target quality value 1,353 may varyover time as requirements change. For example, a customer qualityrequirement may increase or decrease due to customer demands. Thepresent invention allows for rapid on demand updating of the targetquality value 1,353, which in turn will allow the system to immediatelychange the product (sample) selection to bring the average quality ofall products (samples) in collector 1,348 to the desired level. It isnoted herein that collector 1,348 is an optional element and notnecessary. In other embodiments, the samples are alternatively routed tothe remaining processing line.

In one example, the sample sources, the source selecting device(s), theinspection device(s) and the collector are all located at the samefacility so that the product (sample) can travel to each of these partsof the system. However, computing device 1,351 may or may not be locatedat the same facility. In one example, the computing device 1,351 may belocated in close proximity to inspection devices 1,344, 1,345, 1,346 andconnected via an Ethernet connection. In another example, the computingdevice 1,351 may be located on the other side of the world from theinspection devices 1,344, 1,345, 1,346 and connected via a group ofvarious communication technologies, both wired and wireless.

Another note is made with respect to the inspection data 1,340 output byinspection devices. In one example, the inspection devices may onlyoutput captured raw inspection data, such as but not limited to,image(s) 1,349. The captured raw data is then analyzed by computingdevice 1,351 to determine the quality of the product (sample) inspected.In another example, the inspection devices may also analyze the capturedraw data, such as but not limited to, image(s) 1,349 and generateinspection data 1,350, which is output to computing device 1,351. Inthis fashion, the present invention may utilize either advanced smartinspection devices or simple inspection devices that only capture rawdata.

It is also noted herein that computing device 1,351 may be implementedby one or a plurality of computing devices. In one example, computingdevice 1,351 may be a single computer. In another example, computingdevice 1,351 may be implemented utilizing an array of computers orservers that perform various computations of the system.

With respect to all embodiments discussed above, an inspection devicemay be any of the following: an optical sensor, a moisture sensor, amicrotoxin sensor, a thermometer sensor, an acidity sensor, a microwavesensor, a pressure sensor, a level sensor, an ultrasonic sensor, a flowsensor, a viscosity sensor, a conductance/impedance sensor, anelectronic nose (sniffing) sensor, an X-ray sensor, a multi spectral(visual/non visual) sensor, a weight sensor, a refractometer sensor, atenderometer sensor, a firmness sensor, or a hardness sensor.

With respect to all embodiments discussed above, the target qualityvalue may be a single value or more than a single value. For example,the target quality value may be any combination of the following: ashape quality, surface contour quality, hole quality, pest quality, sizequality, moisture level, oil content, fat content, mycotoxin content,broken objects (parts missing), foreign material (such as rocks,plastic, metal, wood, glass, . . . ), discolored objects (partially orcomplete), misshapen objects (not matching a predefined shape), objectsthat do not match certain dimensions (too long, too short, too wide, toonarrow, too thick, to thin, etc.), visual damage (discolored spots orareas, insect damage, shriveled/dried, surface skin damage, mold, decay,or rancid.

With respect to all embodiments discussed above, the computing devicecomprises a processing circuit, a memory unit, and a communication port.The target quality value, the measured quality value, the source qualityvalue, and the source control instruction are communicated via thecommunication port.

With respect to all embodiments discussed above, the system may furtherbe configured to output an alert message when the target quality valueis not achieved. The message may be sent via wired or wireless network.The message may also be sent via an audible message broadcast at theprocessing facility. The message may alternatively be communicated via alocal visual indiator, such as but not limited to, a blinking red light.

With respect to all embodiments discussed above, machine learning may beused to generate the source control instruction. The incoming inspectiondata is fed into a machine learning algorithm on the computing device.Over time the machine learning algorithm builds a model to predict themake up of the product flow. This enables the computing device to takemachine learning driven decisions for the source control instructions.Over time the machine learning model improves and will be capable topredict the future expected quality data more accurately. For example,if the sample sources are fed through bins the bottom of the bin mightcontain more broken product leading to a cyclical pattern in the productstream of increased amounts of broken product at the end of every inputbin. The machine learning appraoch enables the computing device to learnand take into account these cyclical patterns.

FIG. 69 is a flowchart 1,360 of a target quality control system. In step1,361 a target quality value that indicates a desired sample quality ofoutput samples is received. In step 1,362 a measured quality value ofone or more samples is received. In step 1,363 a source quality valueassociated with a source of samples is received. In step 1,364 a sourcecontrol instruction that indicates a source of samples from which futuresamples should be sourced is sent. In step 1,365, the sample from theindicated source is caused to be routed to a collection unit.

FIG. 70 is a flowchart 1,370 of a target quality control system. In step1,371 a quality target value is received. In step 1,372, one or moresource quality values are received. In step 1,373, a source controlinstruction is sent to source selecting device, thereby selecting asample from the first sample source. In step 1,374, the sample from thefirst source of samples is inspected and one or more measured qualityvalues are generated. In step 1,375, the one or more measured qualityvalues are sent to a computing device. In step 1,376, the one or moremeasure quality values are processed. In step 1, 377, a source controlinstruction is generated based on the target quality value, the one ormore source quality values, and the one or more measured quality values.

Product Target Quality Control System with Intelligent Sorting

FIG. 71 is a flowchart diagram of a target quality control system withintelligent inspection and sorting. The system includes one or moreproduct (sample) sources. In one example, the system includes threesources of unknown grade product 1,380, 1,381, and 1,382. The label of“unknown” indicates that the quality, or “grade”, of the product(samples) included in each source are not previously known by thesystem. The quality of the product in each source may also not beuniform; therefore, each source may include wide range of productquality. The system also includes three product inspection and sorters1,383, 1384, and 1,385. Each product inspector and sorter first performan inspection of a product (sample), then sorts the product (sample)depending on the inspection results. For example, if the inspectionindicates that there is a defect in the product, then the product may besorted to a garbage bin. In another example, the product can be sortedfor a different use where a lower quality level product is suitable. Thesystem also includes computing device 1,389. Computing device 1,389 maybe a single computer or an array of computers. The computing device1,389 is configured to send a control command to the product inspector,thereby controlling the method of inspection to be performed by theproduct inspector and sorter. In one example, the control command alsocontrols how the product (sample) is sorted. In another example, theinspection portion of the product inspection and sorter generates acommand controlling how the product (sample) is sorted. After theinspection is complete, the product inspector and sorters send measurequality value(s) to the computing device. The computing device thenprocesses the measured quality value(s) and determines if a new controlcommand is necessary to adjust the quality level of the product(samples) output by the system. In one example, the product output byeach product inspector and sorter is output to separate bins, eachhaving a specific desired quality of product 1,386, 1,387 and 1,388. Itis noted that FIG. 71 is a high-level flowchart provided to give anoverview of the system. Many details and permutations are omitted forsimplicity. A more detailed description of the various embodiments isprovided below.

FIG. 72 is an operation diagram of a first target quality control systemusing intelligent inspection and sorting. The system includes multiplesample sources 1,390, 1,391 and 1,392. Each sample source is configuredto output samples (product) to a different product inspector and sorter1,393, 1,394, and 1,395. The product inspectors and sorters areconfigured to output samples to either a discard bin or to a uniquequality pass collector or processing line 1,397, 1,398, or 1,399. Theysystem also includes a computing device 1,402 that receives a targetquality value 1,403 and measured quality values output from each productinspector and sorter. The measured quality value(s) may includeinspection data 1,401 and may also include images 1,400. In operation,computing device 1,402 outputs command(s) 1,404 to each productinspector and sorter, thereby controlling the manner the method ofinspection and sorting performed by each product inspector and sorter.For example, the command 1,404 may cause a product inspector and sorterto inspect a passing product (sample) and determine if a specific defector characteristic is present in the product. The command 1,404 may causethe product inspector and sorter to direct the outputting of the samplebased upon the inspection results. For example, the inspection mayindicate that a product (sample) has a defect and then accordingly routethe product to a discard bin. Alternatively, the inspection may indicatethat the product has no defects and then accordingly route the productto the passing product collection bin or processing line. It is notedherein that control of the sorting can be performed by either thecomputing device 1,402 (based on the measured quality data) or by theproduct inspector and sorter itself. In the latter scenario, the productinspector and sorter can process the inspection data collected about theproduct and determine the sorting action to be performed on the product.The product inspector may be implemented using any of the inspectordevices described above. For example, the use of one or more cameras maybe used to capture one or more images of the product and then thoseimages may be processed to perform the inspection.

FIG. 73 is an operation diagram of a second target quality controlsystem using intelligent inspection and sorting. The system includes asingle sample source 1,410 which is configured to output samples tomultiple product inspector and sorters 1,411, 1,412 and 1,413. Theproduct inspectors and sorters are configured to output samples toeither a discard bin or to a quality pass collector or processing line1,415. They system also includes a computing device 1,418 that receivesa target quality value 1,419 and measured quality values output fromeach product inspector and sorter. The measured quality value(s) mayinclude inspection data 1,417 and may also include images 1,416. Inoperation, computing device 1,418 outputs command(s) 1,420 to eachproduct inspector and sorter, thereby controlling the manner the methodof inspection and sorting performed by each product inspector andsorter. For example, the command 1,420 may cause a product inspector andsorter to inspect a passing product (sample) and determine if a specificdefect or characteristic is present in the product. The command 1,404may cause the product inspector and sorter to direct the outputting ofthe sample based upon the inspection results. For example, theinspection may indicate that a product (sample) has a defect and thenaccordingly route the product to a discard bin. Alternatively, theinspection may indicate that the product has no defects and thenaccordingly route the product to the passing product collection bin orprocessing line. It is noted herein that control of the sorting can beperformed by either the computing device 1,402 (based on the measuredquality data) or by the product inspector and sorter itself. In thelatter scenario, the product inspector and sorter can process theinspection data collected about the product and determine the sortingaction to be performed on the product. The product inspector may beimplemented using any of the inspector devices described above. Forexample, the use of one or more cameras may be used to capture one ormore images of the product and then those images may be processed toperform the inspection.

A benefit of the product target quality control system with intelligentinspection and sorting system is illuminated in the comparison of FIG.72 with FIG. 73 . The system is capable of outputting product at thedesired target quality level regardless of any preprocessing of thesource samples (product). In other systems, the average quality level ofa sample source is necessary to adjust the output product to a desiredtarget quality level. However, in the present invention the system canprovide a very specific target quality level without any preprocessingof product and without any previous knowledge of each product (sample)source quality levels. This provides a large temporal and financialbenefit to food processors. Preprocessing product requires time andcosts. Removal of preprocessing eliminates this time and costs.

FIG. 74 is a perspective view of an inspection and sorting productionline. The system includes a product sample bin 1,421 that is filled withproduct (samples) 1,422. They system also includes an inspector 1,423and sorter 1,424. An output trough, conveyor, or collection bin 1,425 isalso included in the system. In the present invention, the inspector1,423 and sorter 1,424 are operate together as a single productinspector and sorter. Via a wireless or wired communication medium, theproduct inspector and sorter is in communication with a computingdevice. The computing device can receive a target quality value(s) andoutput a command to the product inspector and sorter, thereby changingthe operation of the product inspector and sorter so that productsatisfying the target quality value(s) is output to the trough,conveyer, collection bin 1,425.

FIG. 74 illustrates multiple inspector-sorter pairs, each of which canbe configured to operate as a product inspection and sorter. Similarly,each of the product inspector and sorter units can be controlled by acomputer device. This configuration allows real-time control of theproduct quality that is output from the system. Further, this systemallows a computing device to record the data for each product (sample)that is inspected. Moreover, the computing device is able to record thesorting operations performed on each product (sample) that passesthrough the system. This data can be of tremendous value. In oneexample, this data is used to determine if the quality of one source ofproduct is significantly better or worse than another source. In anotherexample, this data can be used to determine if they are specific typesof defects in different sample sources. Those benefits are highlydesired in addition to the primary goal of output product at a desiredtarget quality level.

FIG. 75 is an operation diagram of a third target quality control systemusing intelligent inspection and sorting. The system includes multiplesample sources 1,430, 1,431 and 1,432. Each sample source is configuredto output samples (product) to a different product inspectors 1,433,1,434, and 1,435. The product inspectors are configured to output theinspected samples to different sorters 1,436, 1,437, and 1,438. Thesorters are configured to output samples to either a discard bin or to adifferent quality pass collector or processing line 1,440, 1,441, or1,442. They system also includes a computing device 1,445 that receivesa target quality value 1,446 and measured quality values output fromeach product inspector. The measured quality value(s) may includeinspection data 1,444 and may also include images 1,443. In operation,computing device 1,445 outputs command(s) 1,447 to each productinspector and sorter, thereby controlling the manner the method ofinspection and sorting performed by each product inspector and sorter.For example, the command 1,447 may cause a product inspector and sorterto inspect a passing product (sample) and determine if a specific defector characteristic is present in the product. The command 1,447 may causethe product inspector and sorter to direct the outputting of the samplebased upon the inspection results. For example, the inspection mayindicate that a product (sample) has a defect and then accordingly routethe product to a discard bin. Alternatively, the inspection may indicatethat the product has no defects and then accordingly route the productto the passing product collection bin or processing line. It is notedherein that control of the sorting can be performed by either thecomputing device 1,445 (based on the measured quality data) or by theproduct inspector. FIG. 75 illustrates the former. In the latterscenario, the product inspector and sorter can process the inspectiondata collected about the product and determine the sorting action to beperformed on the product. The product inspector may be implemented usingany of the inspector devices described above. For example, the use ofone or more cameras may be used to capture one or more images of theproduct and then those images may be processed to perform theinspection.

FIG. 76 is an operation diagram of a fourth target quality controlsystem using intelligent inspection and sorting. The system includes asingle sample source 1,450. Each sample source is configured to outputsamples (product) to a different product inspectors 1,451, 1,452, and1,453. The product inspectors are configured to output the inspectedsamples to different sorters 1,454, 1,455, and 1,456. The sorters areconfigured to output samples to either a discard bin or to a qualitypass collector or processing line 1,458. They system also includes acomputing device 1,461 that receives a target quality value 1,462 andmeasured quality values output from each product inspector. The measuredquality value(s) may include inspection data 1,460 and may also includeimages 1,459. In operation, computing device 1,461 outputs command(s)1,463 to each product inspector and sorter, thereby controlling themanner the method of inspection and sorting performed by each productinspector and sorter. For example, the command 1,463 may cause a productinspector and sorter to inspect a passing product (sample) and determineif a specific defect or characteristic is present in the product. Thecommand 1,463 may cause the product inspector and sorter to direct theoutputting of the sample based upon the inspection results. For example,the inspection may indicate that a product (sample) has a defect andthen accordingly route the product to a discard bin. Alternatively, theinspection may indicate that the product has no defects and thenaccordingly route the product to the passing product collection bin orprocessing line. It is noted herein that control of the sorting can beperformed by either the computing device 1,461 (based on the measuredquality data) or by the product inspector. FIG. 76 illustrates theformer. In the latter scenario, the product inspector and sorter canprocess the inspection data collected about the product and determinethe sorting action to be performed on the product. The product inspectormay be implemented using any of the inspector devices described above.For example, the use of one or more cameras may be used to capture oneor more images of the product and then those images may be processed toperform the inspection.

FIG. 77 is an operation diagram of a fifth target quality control systemusing intelligent inspection and sorting. The system includes a singlesample source 1,470. Each sample source is configured to output samples(product) to a different product inspectors 1,471, 1,472, and 1,473. Theproduct inspectors are configured to output the inspected samples todifferent sorters 1,474, 1,475, and 1,476. The sorters are configured tooutput samples to either a discard bin or to a quality pass collector orprocessing line 1,478. They system also includes a computing device1,481 that receives a target quality value 1,482 and measured qualityvalues output from each product inspector. The measured quality value(s)may include inspection data 1,480 and may also include images 1,479. Inoperation, computing device 1,481 outputs command(s) 1,483 to eachproduct inspector and sorter, thereby controlling the manner the methodof inspection and sorting performed by each product inspector andsorter. For example, the command 1,483 may cause a product inspector andsorter to inspect a passing product (sample) and determine if a specificdefect or characteristic is present in the product. The command 1,483may cause the product inspector and sorter to direct the outputting ofthe sample based upon the inspection results. For example, theinspection may indicate that a product (sample) has a defect and thenaccordingly route the product to a discard bin. Alternatively, theinspection may indicate that the product has no defects and thenaccordingly route the product to the passing product collection bin orprocessing line. It is noted herein that control of the sorting can beperformed by either the computing device 1,481 (based on the measuredquality data) or by the product inspector. FIG. 77 illustrates thelatter. In the latter scenario, the product inspector and sorter canprocess the inspection data collected about the product and determinethe sorting action to be performed on the product. The product inspectormay be implemented using any of the inspector devices described above.For example, the use of one or more cameras may be used to capture oneor more images of the product and then those images may be processed toperform the inspection.

FIG. 78 is a flowchart 1,490 of a target quality control system usingintelligent inspection and sorting. In step 1,491, a target qualityvalue is received. In step 1,492, an inspection and sort control commandare generated and sent to a product inspector and sorter, therebycontrolling the method of inspection and sorting to be performed onproduct (samples). In step 1,493, a sample is inspected and one or moremeasured quality values are generated. In step 1,494, one or moremeasured quality values from the product inspector and sorter arereceived. In step 1,495, the one or more measured quality values areprocessed. In step 1,496, an inspection and sort control instruction aregenerated based on the target quality value and the one or more measuredquality values, thereby updating the method of inspection and sorting tobe performed.

FIG. 79 is a flowchart 1,500 of a target quality control system usingintelligent inspection and sorting. In step 1,501, a target qualityvalue is received. In step 1,502, an inspection control command isgenerated and sent to a product inspector and sorter, therebycontrolling the method of inspection to be performed on product(samples). In step 1,503, a sort control command is generated and sentto a sorter, thereby controlling the method of sorting to be performedon product (samples). In step 1,504, a sample is inspected and one ormore measured quality values are generated. In step 1,505, one or moremeasured quality values from the product inspector are received. In step1,506, the one or more measured quality values are processed. In step1,507, an inspection control instruction and a sort control instructionare generated based on the target quality value and the one or moremeasured quality values, thereby updating the method of inspection andsorting to be performed.

FIG. 80 is a flowchart 1,510 of a target quality control system usingintelligent inspection and sorting. In step 1,511, a target qualityvalue is received. In step 1,512, an inspection control command isgenerated and sent to a product inspector and sorter, therebycontrolling the method of inspection to be performed on product(samples). In step 1,513, a sample is inspected and one or more measuredquality values are generated. In step 1,514, sort control instruction isgenerated by the product inspector based on the one or more measuredquality values, thereby controlling the method of sorting to beperformed. In step 1,515 the one or more measured quality values arereceived form the product inspector. In step 1, 516, the one or moremeasured quality values are processed. In step 1,517, an inspectioncontrol command is generated based on the target quality value and theone or more measured quality values, thereby updating the method ofinspection to be performed.

FIG. 81 is an operation diagram of a first target quality control systemusing intelligent inspection and sorting as well as output productinspection. The system includes a single sample source 1,520. Eachsample source is configured to output samples (product) to a differentproduct inspectors 1,521, 1,522, and 1,523. The product inspectors areconfigured to output the inspected samples to different sorters 1,524,1,525, and 1,526. The sorters are configured to output the samples todifferent output product inspectors 1,528, 1,529 and 1,530. The outputproduct inspectors are configured to output samples to a quality passcollector or processing line 1,531. They system also includes acomputing device 1,534 that receives a target quality value 1,5352 andmeasured quality values output from each product inspector. The measuredquality value(s) may include inspection data 1,533 and may also includeimages 1,532. Each output product inspector is also configured tocommunicate with the computing device 1,534. The communication may bewired or wireless as described above. The computing device 1,534 isconfigured to output commands 1,540 to control the method of inspectionperformed by each output product inspector. Output product inspectorsare used to measure the passing product (samples) and verify that thedesired target quality level is achieved. In operation, computing device1,534 outputs command(s) 1,536 to each product inspector, therebycontrolling the manner the method of inspection performed by eachproduct inspector and sorter. For example, the command 1,536 may cause aproduct inspector to inspect a passing product (sample) and determine ifa specific defect or characteristic is present in the product. Thecommand 1,536 may cause the product inspector and sorter to direct theoutputting of the sample based upon the inspection results. For example,the inspection may indicate that a product (sample) has a defect andthen accordingly route the product to a discard bin. Alternatively, theinspection may indicate that the product has no defects and thenaccordingly route the product to the passing product collection bin orprocessing line. It is noted herein that control of the sorting can beperformed by either the computing device 1,534 (based on the measuredquality data) or by the product inspector. FIG. 81 illustrates thelatter. In the latter scenario, the product inspector and sorter canprocess the inspection data collected about the product and determinethe sorting action to be performed on the product. The product inspectormay be implemented using any of the inspector devices described above.For example, the use of one or more cameras may be used to capture oneor more images of the product and then those images may be processed toperform the inspection.

FIG. 82 is an operation diagram of a second target quality controlsystem using intelligent inspection and sorting as well as outputproduct inspection. The system includes a single sample source 1,550.Each sample source is configured to output samples (product) to adifferent product inspectors 1,551, 1,552, and 1,553. The productinspectors are configured to output the inspected samples to differentsorters 1,554, 1,555, and 1,556. The sorters are configured to outputthe samples to a single output product inspector 1,558. The outputproduct inspector is configured to output samples to a quality passcollector or processing line 1,559. They system also includes acomputing device 1,562 that receives a target quality value 1,563 andmeasured quality values output from each product inspector. The measuredquality value(s) may include inspection data 1,561 and may also includeimages 1,560. The output product inspector is also configured tocommunicate with the computing device 1,562. The communication may bewired or wireless as described above. The computing device 1,562 isconfigured to output commands 1,568 to control the method of inspectionperformed by each output product inspector. Output product inspectorsare used to measure the passing product (samples) and verify that thedesired target quality level is achieved. In operation, computing device1,562 outputs command(s) 1,564 to each product inspector, therebycontrolling the manner the method of inspection performed by eachproduct inspector and sorter. For example, the command 1,564 may cause aproduct inspector to inspect a passing product (sample) and determine ifa specific defect or characteristic is present in the product. Thecommand 1,564 may cause the product inspector and sorter to direct theoutputting of the sample based upon the inspection results. For example,the inspection may indicate that a product (sample) has a defect andthen accordingly route the product to a discard bin. Alternatively, theinspection may indicate that the product has no defects and thenaccordingly route the product to the passing product collection bin orprocessing line. It is noted herein that control of the sorting can beperformed by either the computing device 1,562 (based on the measuredquality data) or by the product inspector. FIG. 82 illustrates thelatter. In the latter scenario, the product inspector and sorter canprocess the inspection data collected about the product and determinethe sorting action to be performed on the product. The product inspectormay be implemented using any of the inspector devices described above.For example, the use of one or more cameras may be used to capture oneor more images of the product and then those images may be processed toperform the inspection.

FIG. 83 is a flowchart 1,570 of a target quality control system usingintelligent inspection and sorting as well as output product inspection.In step 1,571, a quality target value is received. In step 1,572, aninspection control command is generated and sent to a product inspector,thereby controlling the method of inspection to be performed. In step1,573, an output inspection control instruction is generated and sent anoutput product inspector, thereby controlling the method of outputinspection to be performed. In step 1,574 a sample is inspected and oneor more measured quality values are generated. In step 1,575, a sortcontrol command is generated by the product inspector and sent to thesorter, thereby controlling how the inspected sample is to be sorted. Instep 1,576, one or more measured quality values are received from theproduct inspector and on or more measured quality values are receivedfrom the output product inspector. In step 1,577, the one or moremeasured quality values received from all product inspectors areprocessed. In step 1,578, an inspection control command is generatedbased on the target quality value and the one or more measured qualityvalues received, thereby updating the method of inspection to beperformed.

It is also noted herein that computing device may be implemented by oneor a plurality of computing devices. In one example, the recitedcomputing device may be a single computer. In another example, therecited computing device may be implemented utilizing an array ofcomputers or servers that perform various computations of the system.

With respect to all embodiments discussed above, an inspection devicemay utilize any of the following: an optical sensor, a moisture sensor,a microtoxin sensor, a thermometer sensor, an acidity sensor, amicrowave sensor, a pressure sensor, a level sensor, an ultrasonicsensor, a flow sensor, a viscosity sensor, a conductance/impedancesensor, an electronic nose (sniffing) sensor, an X-ray sensor, a multispectral (visual/non visual) sensor, a weight sensor, a refractometersensor, a tenderometer sensor, a firmness sensor, or a hardness sensor.

With respect to all embodiments discussed above, the target qualityvalue may be a single value or more than a single value. For example,the target quality value may be any combination of the following: ashape quality, surface contour quality, hole quality, pest quality, sizequality, moisture level, oil content, fat content, mycotoxin content,broken objects (parts missing), foreign material (such as rocks,plastic, metal, wood, glass, . . . ), discolored objects (partially orcomplete), misshapen objects (not matching a predefined shape), objectsthat do not match certain dimensions (too long, too short, too wide, toonarrow, too thick, to thin, etc.), visual damage (discolored spots orareas, insect damage, shriveled/dried, surface skin damage, mold, decay,or rancid.

With respect to all embodiments discussed above, the computing devicecomprises a processing circuit, a memory unit, and a communication port.The target quality value, the measured quality value, and the inspectioncontrol instruction are communicated via the communication port.

With respect to all embodiments discussed above, the system may furtherbe configured to output an alert message when the target quality valueis not achieved. The message may be sent via wired or wireless network.The message may also be sent via an audible message broadcast at theprocessing facility. The message may alternatively be communicated via alocal visual indiator, such as but not limited to, a blinking red light.

With respect to all embodiments discussed above, machine learning may beused to generate the inspection control instruction. The incominginspection data is fed into a machine learning algorithm on thecomputing device. Over time the machine learning algorithm builds amodel to predict the make up of the product flow. This enables thecomputing device to take machine learning driven decisions for theinspection control instructions. Over time the machine learning modelimproves and will be capable to predict the future expected quality datamore accurately. For example, if the sample sources are fed through binsthe bottom of the bin might contain more broken product leading to acyclical pattern in the product stream of increased amounts of brokenproduct at the end of every input bin. The machine learning approachenables the computing device to learn and take into account thesecyclical patterns.

The above mentioned sorters may be any sorter known in the art or anysorter described above in the present application, such as but notlimited to, the use of a burst of air to redirect the trajectory of asample as it travels along the processing line, a vacuum that causes thesample to be removed from the flow of samples through the processingline, or the use of a mechanically controlled flap to redirect thesample as it travels along the processing line.

Integrated Adaptable Inspector and Sorting Unit

FIG. 84 is a diagram of a conveyor with an adaptable inspection andsorting unit attached to the conveyor. Adaptable inspection and sortingunit 1,582 is physically mounted to conveyor 1,580. Sample 1,583 iscaused to become in contact with conveyor 1,580. Upon contact, sample1,583 is moved via a rotating conveyor belt of conveyor 1,580. Thefriction between the sample and the rotating conveyor belt causes thesample to be move along the direction of the conveyor belt movement.

The adaptable inspection and sorting unit 1,582 is attached to theconveyor 1,580 via one or more mounting brackets 1,583. One skilled inthe art will readily realize that a various number of brackets andvarious styles of brackets can be used to mount the adaptable inspectionand sorting unit 1,582 to conveyor 1,580. For example, FIG. 87illustrates another bracket geometry that can be utilized to mount theadaptable inspection and sorting unit 1,582 to conveyor 1,580. MountingBracket 1,583 can attach to either the adaptable inspection and sortingunit 1,582 or the conveyor 1,580 using various items, such as bolts,screws, pins, locks, clamps, welds (metals or thermoplastics), adhesive,slots, magnets, rails, gravity, or friction.

The adaptable inspection and sorting unit 1,582 includes an attachmentmechanism, an inspection sensor device (optical receiver), a data portand a power port. The data port and the power port may be combined intoa single physical port that connects to a single cable 1,584 thatincludes both power conductors and data conductors. Alternatively, theadaptable inspection unit 1,582 may include a data port that is separatefrom the power port. Further, the adaptable inspection and sorting unit1,582 may include an antenna connectable data port that connects to anantenna 1,585 so to allow for wireless communication. FIG. 84 does notillustrate the inspection sensor device. FIG. 90 illustrates a blockdiagram of an adaptable inspection unit 1,640 that includes anattachment mechanism 1,641, an inspection sensor device 1,642, a dataport 1,643, a sorting device 1,644, and a power port 1,645.

In operation, the conveyor 1,580 causes the sample 1,581 to travel underthe adaptable inspection and sorting unit 1,582. While the sample is inview of the inspection sensor device that is included in the adaptableinspection and sorting unit 1,582 one or more images of the sample arecaptured and stored in a memory device. The memory device may beincluded in the adaptable inspection and sorting unit 1,582 or may beincluded in a device that communicates with the adaptable inspection andsorting unit 1,582 via the data port (wired or wireless). The capturedsensor data (e.g., images) are then processed by a processor executing aquality inspection algorithm. In one example, the adaptable inspectionunit 1,5582 includes the 3D inspector described in detail above. Inanother example, a 3D image of the sample is generated based on the oneor more images captured by the adaptable inspection and sorting unit1,582. In yet another example, the captured 2D image is used to performthe inspection. The 3D or 2D image(s) are used to determine a qualitycharacteristic of the sample. In one example, the quality characteristicis generated by the adaptable inspection and sorting unit 1,582 andoutput via the data port. In another example, the one or more capturedimages are output from the adaptable inspection and sorting unit 1,582to another device that determines the quality characteristics of thesample. Adaptable inspection and sorting unit 1,582 provides improvequality inspection compared to unreliable inspection by human eyeswithout the cost of replacing an entire processing line. Moreover,adaptable inspection and sorting unit 1,582 is able to inspect many moresamples per unit time than could be inspected by a human.

Absent any sorting mechanism, all samples would be directed toward thesame location regardless of measured quality due to the absence ofsorting functionality. The directing of a sample based upon the measuredquality of the sample is achieved by the adaptable inspection andsorting device 1,582 because the device is capable of sorting samples inaddition to inspecting samples.

In operation, the conveyor 1,580 causes the sample 1,581 to travel underthe adaptable inspection and sorting unit 1,582. While the sample is inreach of the sorting device that is included in the adaptable inspectionand sorting unit 1,582 the sample is sorted as instructed. In oneexample, the sorting instruction is received via the data port andstored in a memory included in the adaptable inspection and sorting unit1,582. In another example, quality characteristic data is received viathe data port and in response the adaptable inspection and sorting unit1,582 generates the sorting instruction. In yet another example, theinformation received via the data port is a percentage of samples to bedeflected. Communication with the adaptable inspection and sorting unit1,582 may be performed via the data port (wired or wireless). Thesorting device may be by a vacuum system, a mechanical pedal system, anair jet system, or a mechanical gate. The adaptable inspection andsorting unit 1,582 performs automated sorting so that high qualitysamples are automatically separated from low quality samples.

Adaptable inspection and sorting unit 1,582 provides improve sortingcompared to unreliable sorting by human hands without the cost ofreplacing an entire processing line. Moreover, adaptable inspection andsorting unit 1,582 is able to sort many more samples per unit time thancould be sorted by a human.

FIG. 85 is a diagram of a conveyor with an adaptable inspection andsorting unit attached to the ceiling above the conveyor. Adaptableinspection and sorting unit 1,592 is physically mounted to the ceilingabove conveyor 1,590. Similar to FIG. 84 , in operation, sample 1,591 iscaused to become in contact with conveyor 1,590. Upon contact, sample1,591 is moved via a rotating conveyor belt of conveyor 1,590. Thefriction between the sample and the rotating conveyor belt causes thesample to be move along the direction of the conveyor belt movement.

The adaptable inspection and sorting unit 1,592 is attached to theceiling above conveyor 1,590 via one or more mounting brackets 1,593.One skilled in the art will readily realize that a various number ofbrackets and various styles of brackets can be used to mount theadaptable inspection and sorting unit 1,592 to the ceiling aboveconveyor 1,590. Mounting Bracket 1,593 can attach to either theadaptable inspection and sorting unit 1,592 or the ceiling aboveconveyor 1,590 using various items, such as bolts, screws, pins, locks,clamps, welds (metals or thermoplastics), adhesive, slots, magnets,rails, gravity, or friction.

The adaptable inspection and sorting unit 1,592 includes an attachmentmechanism, an inspection sensor device (optical receiver), a data portand a power port. The data port and the power port may be combined intoa single physical port that connects to a single cable 1,594 thatincludes both power conductors and data conductors. Alternatively, theadaptable inspection and sorting unit 1,592 may include a data port thatis separate from the power port. Further, the adaptable inspection andsorting unit 1,592 may include an antenna connectable data port thatconnects to an antenna 1,595 so to allow for wireless communication.FIG. 85 does not illustrate the inspection sensor device. FIG. 90illustrates a block diagram of an adaptable inspection and sorting unit1,640 that includes an attachment mechanism 1,641, an inspection sensordevice 1,642, a data port 1,643, a sorting device 1,644, and a powerport 1,645

In operation, the conveyor 1,590 causes the sample 1,591 to travel underthe adaptable inspection and sorting unit 1,592. While the sample is inview, or reach, of the inspection sensor device that is included in theadaptable inspection and sorting unit 1,592 one or more characteristicsand/or images of the sample are captured and stored in a memory device.The memory device may be included in the adaptable inspection andsorting unit 1,592 or may be included in a device that communicates withthe adaptable inspection and sorting unit 1,592 via the data port (wiredor wireless). The captured characteristics and/or image(s) are thenprocessed by a processor executing a quality inspection algorithm. Inone example, the adaptable inspection and sorting unit 1,592 includesthe 3D inspector described in detail above. In another example, a 3Dimage of the sample is generated based on the one or more imagescaptured by the adaptable inspection and sorting unit 1,592. In yetanother example, the captured 2D image is used to perform theinspection. The 3D and/or 2D image(s) are used to determine a qualitycharacteristic of the sample. In one example, the quality characteristicis generated by the adaptable inspection and sorting unit 1,592 andoutput via the data port. In another example, the one or more capturedimages are output from the adaptable inspection and sorting unit 1,592to another device that determines the quality characteristics of thesample. Adaptable inspection and sorting unit 1,592 provides improvequality inspection compared to unreliable inspection by human eyeswithout the cost of replacing an entire processing line. Moreover,adaptable inspection and sorting unit 1,592 is able to inspect many moresamples per unit time than could be inspected by a human.

Absent any sorting mechanism, all samples would be directed toward thesame location regardless of measured quality due to the absence ofsorting functionality. The directing of a sample based upon the measuredquality of the sample is achieved by the adaptable inspection andsorting device 1,592 because the device is capable of sorting samples inaddition to inspecting samples.

In operation, the conveyor 1,590 causes the sample 1,591 to travel underthe adaptable inspection and sorting unit 1,592. While the sample is inreach of the sorting device that is included in the adaptable inspectionand sorting unit 1,592 the sample is sorted as instructed. In oneexample, the sorting instruction is received via the data port andstored in a memory included in the adaptable inspection and sorting unit1,592. In another example, quality characteristic data is received viathe data port and in response the adaptable inspection and sorter unit1,592 generates the sorting instruction. In yet another example, theinformation received via the data port is a percentage of samples to bedeflected. Communication with the adaptable inspection and sorting unit1,592 may be performed via the data port (wired or wireless). Thesorting device may be a vacuum system, a mechanical pedal system, an airjet system, or a mechanical gate. The adaptable inspection and sortingunit 1,592 performs automated sorting so that high quality samples areautomatically separated from low quality samples.

Adaptable inspection and sorting unit 1,592 provides improve sortingcompared to unreliable sorting by human hands without the cost ofreplacing an entire processing line. Moreover, adaptable inspection andsorting unit 1,592 is able to sort many more samples per unit time thancould be sorted by a human.

FIG. 86 is a diagram of a conveyor with an adaptable inspection andsorting unit attached to the floor below the conveyor. Adaptableinspection and sorting unit 1,602 is physically mounted to the floorbelow conveyor 1,600. Similar to FIG. 85 , in operation, sample 1,601 iscaused to become in contact with conveyor 1,600. Upon contact, sample1,601 is moved via a rotating conveyor belt of conveyor 1,600. Thefriction between the sample and the rotating conveyor belt causes thesample to be move along the direction of the conveyor belt movement.

The adaptable inspection and sorting unit 1,602 is attached to the floorbelow conveyor 1,600 via one or more mounting bracket 1,603 and mountingstand 1,606. One skilled in the art will readily realize that a variousnumber of brackets and various styles of brackets can be used to mountthe adaptable inspection and sorting unit 1,602 to the floor belowconveyor 1,600. Mounting Bracket 1,603 can attach to either theadaptable inspection and sorting unit 1,602 or the floor below conveyor1,600 using various items, such as bolts, screws, pins, locks, clamps,welds (metals or thermoplastics), adhesive, slots, magnets, rails,gravity, or friction.

The adaptable inspection and sorting unit 1,600 includes an attachmentmechanism, an inspection sensor device (optical receiver), a data portand a power port. The data port and the power port may be combined intoa single physical port that connects to a single cable 1,604 thatincludes both power conductors and data conductors. Alternatively, theadaptable inspection and sorting unit 1,602 may include a data port thatis separate from the power port. Further, the adaptable inspection andsorting unit 1,602 may include an antenna connectable data port thatconnects to an antenna 1,605 so to allow for wireless communication.FIG. 86 does not illustrate the inspection sensor device. FIG. 90illustrates a block diagram of an adaptable inspection and sorting unit1,640 that includes an attachment mechanism 1,641, an inspection sensordevice 1,642, a data port 1,643, a sorting device 1,644, and a powerport 1,645.

In operation, the conveyor 1,600 causes the sample 1,601 to travel underthe adaptable inspection and sorting unit 1,602. While the sample is inview, or reach, of the inspection sensor device that is included in theadaptable inspection and sorting unit 1,602 one or more characteristicsand/or images of the sample are captured and stored in a memory device.The memory device may be included in the adaptable inspection andsorting unit 1,602 or may be included in a device that communicates withthe adaptable inspection and sorting unit 1,602 via the data port (wiredor wireless). The captured characteristics and/or image(s) are thenprocessed by a processor executing a quality inspection algorithm. Inone example, the adaptable inspection and sorting unit 1,602 includesthe 3D inspector described in detail above. In another example, a 3Dimage of the sample is generated based on the one or more imagescaptured by the adaptable inspection and sorting unit 1,602. In yetanother example, the captured 2D image is used to perform theinspection. The 3D and/or 2D image(s) are used to determine a qualitycharacteristic of the sample. In one example, the quality characteristicis generated by the adaptable inspection and sorting unit 1,602 andoutput via the data port. In another example, the one or more capturedimages are output from the adaptable inspection and sorting unit 1,602to another device that determines the quality characteristics of thesample.

Adaptable inspection and sorting unit 1,602 provides improve qualityinspection compared to unreliable inspection by human eyes without thecost of replacing an entire processing line. Moreover, adaptableinspection and sorting unit 1,602 is able to inspect many more samplesper unit time than could be inspected by a human.

Absent any sorting mechanism, all samples would be directed toward thesame location regardless of measured quality due to the absence ofsorting functionality. The directing of a sample based upon the measuredquality of the sample is achieved by the adaptable inspection andsorting device 1,602 because the device is capable of sorting samples inaddition to inspecting samples.

In operation, the conveyor 1,600 causes the sample 1,601 to travel underthe adaptable inspection and sorting unit 1,602. While the sample is inreach of the sorting device that is included in the adaptable inspectionand sorting unit 1,602 the sample is sorted as instructed. In oneexample, the sorting instruction is received via the data port andstored in a memory included in the adaptable inspection and sorting unit1,602. In another example, quality characteristic data is received viathe data port and in response the adaptable inspection and sorter unit1,602 generates the sorting instruction. In yet another example, theinformation received via the data port is a percentage of samples to bedeflected. Communication with the adaptable inspection and sorting unit1,602 may be performed via the data port (wired or wireless). Thesorting device may be a vacuum system, a mechanical pedal system, an airjet system, or a mechanical gate. The adaptable inspection and sortingunit 1,602 performs automated sorting so that high quality samples areautomatically separated from low quality samples.

Adaptable inspection and sorting unit 1,602 provides improve sortingcompared to unreliable sorting by human hands without the cost ofreplacing an entire processing line. Moreover, adaptable inspection andsorting unit 1,602 is able to sort many more samples per unit time thancould be sorted by a human.

FIG. 88 is a diagram of a conveyor with an adaptable inspection andsorting unit attached to a mounting stand. Adaptable inspection andsorting unit 1,622 has been physically mounted to a mounting stand 1,626located next to conveyor 1,620. Similar to FIG. 87 , in operation,sample 1,621 is caused to become in contact with conveyor 1,620. Uponcontact, sample 1,621 is moved via a rotating conveyor belt of conveyor1,620. The friction between the sample and the rotating conveyor beltcauses the sample to be move along the direction of the conveyor beltmovement.

The adaptable inspection and sorting unit 1,622 is attached to themounting stand 1,626, located next to conveyor 1,620, via one or moremounting brackets 1,623. One skilled in the art will readily realizethat a various number of brackets and various styles of brackets can beused to mount the adaptable inspection and sorting unit 1,622 to themounting stand 1,626. Mounting Bracket 1,623 can attach to either theadaptable inspection and sorting unit 1,622 or the mounting stand 1,626using various items, such as bolts, screws, pins, locks, clamps, welds(metals or thermoplastics), adhesive, slots, magnets, rails, gravity, orfriction.

The adaptable inspection and sorting unit 1,622 includes an attachmentmechanism, an inspection sensor device (e.g., optical receiver), a dataport and a power port. The data port and the power port may be combinedinto a single physical port that connects to a single cable 1,624 thatincludes both power conductors and data conductors. Alternatively, theadaptable inspection and sorting unit 1,622 may include a data port thatis separate from the power port. Further, the adaptable inspection andsorting unit 1,622 may include an antenna connectable data port thatconnects to an antenna 1,625 so to allow for wireless communication.FIG. 88 does not illustrate the inspection sensor device. FIG. 90illustrates a block diagram of an adaptable inspection and sorting unit1,640 that includes an attachment mechanism 1,641, an inspection sensordevice 1,642, a data port 1,643, a sorting device 1,644, and a powerport 1,645.

In operation, the conveyor 1,620 causes the sample 1,621 to travel underthe adaptable inspection and sorting unit 1,622. While the sample is inview, or reach, of the inspection sensor device that is included in theadaptable inspection and sorting unit 1,622 one or more characteristicsand/or images of the sample are captured and stored in a memory device.The memory device may be included in the adaptable inspection andsorting unit 1,622 or may be included in a device that communicates withthe adaptable inspection and sorting unit 1,622 via the data port (wiredor wireless). The captured characteristics and/or image(s) are thenprocessed by a processor executing a quality inspection algorithm. Inone example, the adaptable inspection and sorting unit 1,622 includesthe 3D inspector described in detail above. In another example, a 3Dimage of the sample is generated based on the one or more imagescaptured by the adaptable inspection and sorting unit 1,622. In yetanother example, the captured 2D image is used to perform theinspection. The 3D and/or 2D image(s) are used to determine a qualitycharacteristic of the sample. In one example, the quality characteristicis generated by the adaptable inspection and sorting unit 1,622 andoutput via the data port. In another example, the one or more capturedimages are output from the adaptable inspection and sorting unit 1,622to another device that determines the quality characteristics of thesample.

Adaptable inspection and sorting unit 1,622 provides improve qualityinspection compared to unreliable inspection by human eyes without thecost of replacing an entire processing line. Moreover, adaptableinspection and sorting unit 1,622 is able to inspect many more samplesper unit time than could be inspected by a human.

Absent any sorting mechanism, all samples would be directed toward thesame location regardless of measured quality due to the absence ofsorting functionality. The directing of a sample based upon the measuredquality of the sample is achieved by the adaptable inspection andsorting device 1,622 because the device is capable of sorting samples inaddition to inspecting samples.

In operation, the conveyor 1,630 causes the sample 1,621 to travel underthe adaptable inspection and sorting unit 1,622. While the sample is inreach of the sorting device that is included in the adaptable inspectionand sorting unit 1,622 the sample is sorted as instructed. In oneexample, the sorting instruction is received via the data port andstored in a memory included in the adaptable inspection and sorting unit1,622. In another example, quality characteristic data is received viathe data port and in response the adaptable inspection and sorter unit1,622 generates the sorting instruction. In yet another example, theinformation received via the data port is a percentage of samples to bedeflected. Communication with the adaptable inspection and sorting unit1,622 may be performed via the data port (wired or wireless). Thesorting device may be a vacuum system, a mechanical pedal system, an airjet system, or a mechanical gate. The adaptable inspection and sortingunit 1,622 performs automated sorting so that high quality samples areautomatically separated from low quality samples.

Adaptable inspection and sorting unit 1,622 provides improve sortingcompared to unreliable sorting by human hands without the cost ofreplacing an entire processing line. Moreover, adaptable inspection andsorting unit 1,622 is able to sort many more samples per unit time thancould be sorted by a human.

FIG. 89 is a diagram of a conveyor with an adaptable inspection andsorting unit attached to the conveyor sidewall. The adaptable inspectionand sorting unit can be attached permanently or temporarily to theconveyor sidewall. Conveyor 1,630 includes one or more sidewalls 1,631and a belt that rotates about two or more pulleys. The sidewall 1,631 isincluded in the conveyor 1,630 so to prevent samples from falling offthe sides of the conveyor 1,630. The sidewall 1,631 of the conveyor1,630 can be used to support the adaptable inspection and sorting unit1,632.

The adaptable inspection and sorting unit 1,632 can be attached usingmany different mechanisms. Some of these mechanisms are listed on FIG.89 . These attachment mechanisms include welding the adaptableinspection and sorting unit 1,632 to the conveyor sidewall 1,631, gluingthe adaptable inspection and sorting unit 1,632 to the conveyor sidewall1,631, clamping the adaptable inspection and sorting unit 1,632 to theconveyor sidewall 1,631, magnetically attracting the adaptableinspection unit 782 to the conveyor sidewall 1,631, latching theadaptable inspection and sorting unit 1,632 to the conveyor sidewall1,631, locking the adaptable inspection and sorting unit 1,632 to theconveyor sidewall 1,631, location pinning the adaptable inspection unit782 to the conveyor sidewall 1,631, rail mating the adaptable inspectionand sorting unit 1,632 to the conveyor sidewall 1,631, slide fitting theadaptable inspection and sorting unit 1,632 to the conveyor sidewall1,631, lock pinning the adaptable inspection and sorting unit 1,632 tothe conveyor sidewall 1,631, or using gravity and friction to “attach”the adaptable inspection and sorting unit 1,632 to the conveyor sidewall1,631.

The adaptable inspection and sorting unit 1,632 includes an attachmentmechanism, an inspection sensor device (e.g., an optical receiver), adata port and a power port. The data port and the power port may becombined into a single physical port that connects to a single cablethat includes both power conductors and data conductors. Alternatively,the adaptable inspection and sorting unit 1,632 may include a data portthat is separate from the power port. Further, the adaptable inspectionand sorting unit 1,632 may include an antenna connectable data port thatconnects to an antenna so to allow for wireless communication. FIG. 89does not illustrate the inspection sensor device. FIG. 90 illustrates ablock diagram of an adaptable inspection and sorting unit 1,640 thatincludes an attachment mechanism 1,641, an inspection sensor device1,642, a data port 1,643, a sorting device 1,644, and a power port1,645.

In operation, the conveyor 1,630 causes the sample 1,631 to travel underthe adaptable inspection and sorting unit 1,632. While the sample is inview, or reach, of the inspection sensor device that is included in theadaptable inspection and sorting unit 1,632 one or more characteristicsand/or images of the sample are captured and stored in a memory device.The memory device may be included in the adaptable inspection andsorting unit 1,632 or may be included in a device that communicates withthe adaptable inspection and sorting unit 1,632 via the data port (wiredor wireless). The captured characteristics and/or image(s) are thenprocessed by a processor executing a quality inspection algorithm. Inone example, the adaptable inspection and sorting unit 1,632 includesthe 3D inspector described in detail above. In another example, a 3Dimage of the sample is generated based on the one or more imagescaptured by the adaptable inspection and sorting unit 1,632. In yetanother example, the captured 2D image is used to perform theinspection. The 3D and/or 2D image(s) are used to determine a qualitycharacteristic of the sample. In one example, the quality characteristicis generated by the adaptable inspection and sorting unit 1,632 andoutput via the data port. In another example, the one or more capturedimages are output from the adaptable inspection and sorting unit 1,632to another device that determines the quality characteristics of thesample. Adaptable inspection and sorting unit 1,632 provides improvequality inspection compared to unreliable inspection by human eyeswithout the cost of replacing an entire processing line. Moreover,adaptable inspection and sorting unit 1,632 is able to inspect many moresamples per unit time than could be inspected by a human.

Absent any sorting mechanism, all samples would be directed toward thesame location regardless of measured quality due to the absence ofsorting functionality. The directing of a sample based upon the measuredquality of the sample is achieved by the adaptable inspection andsorting device 1,632 because the device is capable of sorting samples inaddition to inspecting samples.

In operation, the conveyor 1,630 causes the sample 1,631 to travel underthe adaptable inspection and sorting unit 1,632. While the sample is inreach of the sorting device that is included in the adaptable inspectionand sorting unit 1,632 the sample is sorted as instructed. In oneexample, the sorting instruction is received via the data port andstored in a memory included in the adaptable inspection and sorting unit1,632. In another example, quality characteristic data is received viathe data port and in response the adaptable inspection and sorter unit1,632 generates the sorting instruction. In yet another example, theinformation received via the data port is a percentage of samples to bedeflected. Communication with the adaptable inspection and sorting unit1,632 may be performed via the data port (wired or wireless). Thesorting device may be a vacuum system, a mechanical pedal system, an airjet system, or a mechanical gate. The adaptable inspection and sortingunit 1,632 performs automated sorting so that high quality samples areautomatically separated from low quality samples.

Adaptable inspection and sorting unit 1,632 provides improve sortingcompared to unreliable sorting by human hands without the cost ofreplacing an entire processing line. Moreover, adaptable inspection andsorting unit 1,632 is able to sort many more samples per unit time thancould be sorted by a human.

FIG. 91 is a flowchart 1,650 illustrating the operations performed by anadaptable inspection and sorting unit. In step 1,651, an attachmentmechanism is connected to the adaptable inspection and sorting unit. Instep 1,652, the attachment mechanism is connected to the existingprocessing line. This can be a connection directly to the existingprocessing line or to an object near the existing processing line, suchas a wall, ceiling, mounting stand, or conveyor sidewall. In step 1,653,a power port of the adaptable inspection and sorting unit is connectedto a power source. In step 1,654, a data port of the adaptableinspection and sorting unit is connected to a data communicationchannel. The data communication channel can be a wired or wirelesschannel. In step 1,655, the existing processing line is run with theadaptable inspection and sorting unit in place and executing. In step1,656, the existing processing line equipment is capable of performingautomated inspection and sorting.

Proper alignment between the adaptable inspection and sorting unit andthe conveyor is necessary to ensure proper operation. If the position ofthe adaptable inspection and sorting unit moved relative to the positionof the conveyor belt, the adaptable inspection and sorting unit may nolonger be able to properly inspect or sort the samples traveling alongthe conveyor. Accordingly, the mechanisms described above are veryimportant and useful to ensure proper operation.

As discussed above regarding inspecting and sorting operations, machinelearning algorithms may be used to conduct inspection and sortingoperations. For example, a machine learning neural network may be usedto train a computer system to analyze inspection data and create sortinginstructions based on learned data.

Another import aspect required for proper operation of the adaptableinspection and sorter unit is measuring the speed of the sample movingalong the conveyor. The speed at which the sample is traveling is neededto ensure actions made by the adaptable inspection and sorting deviceare conducted at the correct time. Conveyor speeds may vary across timeand across different conveyors. For example, if one or more images aretaken of a sample at time T0 and analysis of the image(s) determine thatthe sample is to be sorted to a garbage bin, the adaptable inspectionand sorting unit needs to know when the sorting operation should beconducted. This can be calculated by adaptable inspection and sortingunit once the speed at which the sample is moving is known. For example,if the sample moves between the inspection location and the sortinglocation in T1 seconds, then the adaptable inspection and sorting unitwill conduct the sorting operation at T1 seconds after the inspectionoperation was conducted (TO). Sample speed can be measured using variousmethods. One such method includes capturing two consecutive images ofthe sample as it moves along the conveyor. Then the movement of thesample is measured by measuring the difference in pixel locations wherethe sample is present. The distance per pixel of the image is known bythe system. Therefore, the system can calculate the speed of the sampleby multiplying (i) the number of pixels the sample moved between the twosequential images by the known distance per pixel, and (ii) thendividing the calculated distance the sample moved by the time durationbetween the first and second sequential images. Once the sample speedalong the conveyor is known, the adaptable inspection and sorting unitcan properly set the delay, or time duration, between conducting theinspection operation and conducting the sorting operation.

An integrated adaptable inspection and sorting unit provides a solutionfor an important market need. First, there is a market need to add aninspection unit or a sorting unit to an existing processing line so toallow current processing operations to continue the use of the existingprocessing line system while adding the clear benefits of automatedinspection and sorting with minimal cost (new machinery and installationlabor). Second, there is a market need to add an inspection unit and asorting unit in an efficient and reliable manner. The alignment of aninspection unit and a sorting unit relative to the processing line (e.g.conveyor belt) is very important to attain optimum operation.Additionally, the alignment between the inspection unit and the sortingunit is very important as well to ensure that proper sorting can beachieved. When a standalone inspection unit is installed next to a standalone sorting unit, effort and time needs to be consumed to ensure theproper alignment of both the inspection unit and the sorting unit,relative to the processing line as well as to each other. Theintegration of the inspection unit and the sorting unit into a singleadaptable unit not only greatly reduces the installation time and cost,the integration into a single unit provides consistent high qualityalignment between the inspector unit and the sorting unit for everyinstallation. This is the case because the integrated adaptableinspection and sorting unit has fixed processing line where each unit isconsistently aligned, checked, and fixed into place before shipping outto various installation locations. This high quality consistentalignment provides improved consistent performance as well as lowerinstallation and maintenance costs.

Automated Inspection Data Collection for Machine Learning Applications

In the ever growing field of artificial intelligence and machinelearning there is a growing need for data, however, not all data is thesame. Data can be of different quality values, judged on an infinitevariety of dimensions. In the space of optical inspection there is aneed for high quality inspection data to aid machine learning (“TrainingData”) for improved artificial intelligence systems that optimize theoperation of optical inspectors. One type of training data that ishighly valuable to a machine learning optical inspector system is dataincluding images of samples that result in a low confidencedetermination by the machine learning system. The data resulting in lowconfidence determinations are the areas where the machine learningsystems have the greatest room for improvement. How to gather, store,and utilize this low confidence data is a problem that currently needs asolution. Multiple solutions to this problem are provided below.

FIG. 92 is a flowchart 1,660 illustrating a first method of automatedinspection data collection for machine learning applications. In step1,661, measurement data is collected. Measurement data may include acaptured image or any output provides by any type of sensor. Anyexemplary list of sensors include: an optical sensor, a moisture sensor,a microtoxin sensor, a thermometer sensor, an acidity (pH) sensor, amicrowave sensor, a pressure sensor, a level sensor, an ultrasonicsensor, a flow sensor, a viscosity sensor, a conductance/impedancesensor, an electronic nose (sniffing) sensor, an x-ray sensor, a multiSpectral (visual/non visual) sensor, a weight sensor, a refractometersensor, a tenderometer sensor, a firmness sensor, a hardness sensors, ora proximity sensor. In step, 1,662, a confidence value associated withthe measurement data is determined. In step, 1,663, a determination ismade as to whether the confidence value is less than a confidencethreshold value. In step, 1,664, the measurement data is caused to bestored in a memory device if the confidence value is less than theconfidence threshold value.

FIG. 93 is a flowchart 1,670 illustrating a second method of automatedinspection data collection for machine learning applications. In step1,671, measurement data is collected. In step, 1,672, a confidence valueassociated with the measurement data is determined. In step, 1,673, adetermination is made as to whether the confidence value is less than aconfidence threshold value. In step, 1,674, the measurement data iscaused to be stored in a memory device if the confidence value is lessthan the confidence threshold value. In step, 1,675, the measurementdata is utilized for training of a machine learning system, therebyupdating the operation of the machine learning system.

A confidence value is a measure of how confident the machine learningsystem is that the observed characteristic of the sample is in factpresent on the sample. For example, the machine learning system maydetermine that a first sample has a specific type of defect, however,the machine learning system may only be fifty percent certain that thedetected defect is present on the first sample. Alternatively, uponinspection of a second sample, the machine learning system maybe ninetypercent certain that the detected defect is present on the secondsample. In one embodiment, the confidence value threshold maybe sixtypercent and therefore the measurement data of the first sample would bedeemed a low confidence measurement data, while the measurement data ofthe second sample would not be deemed a low confidence measurement data.

In one example, the steps illustrated in FIG. 93 may be used to updateinspector operating instructions. The machine learning system utilizesthe measurement data to train the machine learning system how toidentify and determine the characteristics included in the measurementdata. After the training of the machine learning system is completed,updated inspector operating instructions are generated. The updatedinspector operating instructions will provide improved determination ofinspected samples, such that, other samples with similar characteristicswill not longer result in a low confidence value, but rather a highconfidence value because the system is now trained to detect andidentify the characteristics observed in the measurement data.

FIG. 94 is a flowchart 1,680, illustrating a first method of determininga confidence value threshold for automated inspection data collectionfor machine learning applications. In step 1,681, measurement data of acontrol sample is collected. The characteristics of the control sampleis known. In step 1,682, a confidence value associated with themeasurement data is determined. The confidence value indicates theprobability that the measurement data is the same as the knowncharacteristics of the sample. In step 1,683, the measurement data iscompared to the known characteristics of the sample. In step 1,684, aconfidence value threshold is set to a confidence value where themeasurement data is not the same as the known characteristics of thesample. In one example, the control sample may be a type of nut, such asan almond.

FIG. 95 is a flowchart 1,690 illustrating a second method of determininga confidence value threshold for automated inspection data collectionfor machine learning applications. In step 1,691, measurement data of acontrol sample is collected. The characteristics of the control sampleis known. In step 1,692, a confidence value associated with themeasurement data is determined. The confidence value indicates theprobability that the measurement data is the same as the knowncharacteristics of the sample. In step 1,693, the measurement data iscompared to the known characteristics of the sample. In step 1,694, aconfidence value threshold is set to a confidence value where themeasurement data is not the same as the known characteristics of thesample. In step 1,695, steps 1,691 through step 1,694 are repeated usinga plurality of samples, where the characteristics of each of theplurality of samples is known. Thereafter, an average of all resultingconfidence value thresholds is calculated to determine an aggregateconfidence value threshold.

FIG. 96 is a system diagram of a first system configured to performautomated inspection data collection for machine learning applications.Inspector 1,700 collects measurement data of a sample and communicatesthe measurement data to a computing system 1,701. In one example, theinspector 1,700 sends all measurement data to the computing system1,701. In another example, the inspector 1,700 only sends measurementdata determined to be low confidence measurement data to the computingsystem 1,701. The computing system 1,701 then utilized the receivedmeasurement data 1,702 to perform machine learning, thereby updating theinspector operating instructions 1,703. The updated inspector operatinginstructions 1,703 are then communicated to the inspector 1,700. Theinspector 1,700 then begins operation utilizing the updated inspectoroperating instructions 1,703. The communications between the inspector1,700 and the computing system 1,701 may performed via a wired orwireless medium. An example of a wired medium is an ethernet connection.An example of a wireless medium is a WiFi connection. The inspector maybe any type of inspector configured to inspect samples. In oneembodiment, the inspector is an in-flight 3D inspector. In anotherembodiment, the inspector is an adaptable inspection unit.

The system illustrated in FIG. 96 has the advantage of using largeamounts of server processing power to conduct the machine learningoperation on the low confidence measurement data 1,702. Due to cost,size, and power consumption restraints, the amount of processing poweravailable in a single inspector 1,700 is not as great as the processingpower that is available in a computing system 1,701 including one ormore servers.

Local determination of low confidence measurement data by the inspector1,700 has the advantage of reducing the amount of data communicatedbetween the inspector 1,700 and the computing system 1,701. Reduced datacommunication allows for a lower data bandwidth connection requirementbetween the inspector 1,700 and the computing system 1,701. Not only arelower data bandwidth connections less expensive, in many inspectionlocations a high data bandwidth connection is not available.

FIG. 97 is a system diagram of a second system configured to performautomated inspection data collection for machine learning applications.Inspector 1,710 collects measurement data of a sample and processes themeasurement data locally. Measurement data 1,712 is processed by one ormore processors 1,713. The one or more processors 1,713 determine if themeasurement data is low confidence measurement data. Low confidencemeasurement data 1,714 is written to memory space 1,716 located inmemory 1,715. The one or more processors 1,713 read the low confidencemeasurement data 1,714 from memory 1,711 and perform machine learning togenerate updated inspector operating instructions 1,718. The updatedinspector operating instructions 1,718 are written to the operatingsystem memory space 1,717, thereby updating the operation of theinspector 1,710. In one example, the low confidence measurement datamemory space 1,716 and the operating system memory space 1,717 are bothlocated within memory 1,715. In another example, the low confidencemeasurement data memory space 1,716 and the operating system memoryspace 1,717 are located in separate memory devices.

One advantage of the stand alone inspector is that external datacommunication is not necessary. The inspector can operate in solitudeand continually learn and update its operating instructionsindependently, without the need for communication or remote computersystem services.

It is also noted herein, the that the updated operating instructions canbe communicated with more than a single inspector. For example,measurement data from a first inspector may be used to for machinelearning to generate an updated inspector operating instructions whichare then communicated to one or more other inspector units that did notgenerate the measurement data. In this fashion, an array of inspectorunits could benefit from the machine learning resulting from measurementdata collected from a single inspector unit. One skilled in the art,will readily understand the magnitude of this novel aspect whenconsidering a system including many different inspector units, eachgenerating high quality low confidence measurement data to the machinelearning application.

The multiple solutions described above provide structures and processesfor efficiently generating high quality data for machine learningapplications. More specifically, these structures and processes can beused to improve machine learning application within the field ofinspection devices.

Automated Sample Weight Measurement Via Optical Inspection

In the field of product inspection, measuring the quality and quantityof the product is very valuable. In addition to measuring the quantityin sheer numbers of units, it is also valuable to measure the quantityin terms of weight, both per unit and overall total weight of all theproduct. For example, in the field of almond inspection, it is valuableto measure the maximum, minimum, average weight of a group of almonds,and the total sum weight of all almonds in the group. This type of datacoupled with the quality data provides great insight into the finalvalue of the group of inspected almonds. This holds true with respect toinspection of other food and products. A method for automaticallymeasuring sample weight via optical inspection is provided below.

FIG. 98 is a diagram illustrating sample size measurement along a firstplane by pixel counting. The grid is pixel array 1,720 of a capturedimage. The pixels are part of an image captured in the x-y dimension.The shaded squares are pixels displaying a portion of the sample. Theunshaded squares are pixels that are not displaying a portion of thesample. This figure illustrates a method of determining the area of thesample in the x-y plane by counting the shaded pixels that display aportion of the sample. The counted number of shaded pixels multiplied bythe area displayed in each pixel is the approximate area of the samplein the x-y plane.

FIG. 99 is a diagram illustrating sample size measurement along a firstplane by pixel bounding box calculation. The grid is pixel array 1,730of a captured image. The pixels are part of an image captured in the x-ydimension. The shaded squares are pixels displaying a portion of thesample. The unshaded squares are pixels that are not displaying aportion of the sample. This figure illustrates a method of determiningthe area of the sample in the x-y plane by creating a bounding box 1,731around the shaded pixels that display a portion of the sample. Thenumber of pixels inside the bounding box multiplied by the areadisplayed in each pixel is the approximate area of the sample in the x-yplane. One skilled in the art will realize that use of a bounding box,in certain scenarios, can be less accurate than counting pixels todetermine sample area.

FIG. 100 is a diagram illustrating sample size measurement along asecond plane by pixel counting. The grid is pixel array 1,740 of acaptured image. The pixels are part of an image captured in the x-zdimension. The shaded squares are pixels displaying a portion of thesample. The unshaded squares are pixels that are not displaying aportion of the sample. This figure illustrates a method of determiningthe area of the sample in the x-z plane by counting the shaded pixelsthat display a portion of the sample. The counted number of shadedpixels multiplied by the area displayed in each pixel is the approximatearea of the sample in the x-z plane.

FIG. 101 is a diagram illustrating sample area measurement along asecond plane by pixel bounding box calculation. The grid is pixel array1,750 of a captured image. The pixels are part of an image captured inthe x-z dimension. The shaded squares are pixels displaying a portion ofthe sample. The unshaded squares are pixels that are not displaying aportion of the sample. This figure illustrates a method of determiningthe area of the sample in the x-z plane by creating a bounding box 1,751around the shaded pixels that display a portion of the sample. Thenumber of pixels inside the bounding box multiplied by the areadisplayed in each pixel is the approximate area of the sample in the x-zplane. One skilled in the art will realize that use of a bounding box,in certain scenarios, can be less accurate than counting pixels todetermine sample area.

FIG. 102 is a flowchart 1,760 illustrating the steps of automated samplearea measurement. In step 1,761, measurement data of the sample iscollected. In step 1,762, a characteristic of the measurement data isdetermined. In step 1,763, an approximate area of the sample iscalculated based at least in part on the characteristic of the measureddata. In one example, the measured data includes a captured image. Inanother example, the characteristic of the measured data is one or moredimensions of the sample displayed in the captured image. In oneembodiment, the measurement data is collected while the sample isin-flight utilizing an in-flight 3D inspector. In another embodiment,the measurement data is collected by an adaptable inspection unit.

FIG. 103 is a flowchart 1,770 illustrating the steps of automated samplearea measurement. In step 1,771, an image of the sample is captured. Instep 1,772, the number of pixels in the captured image that display aportion of the sample is determined. In step 1,773, the number of pixelsin the captured image that display a portion of the sample aremultiplied by the area displayed in each pixel to calculate theapproximate area of the sample.

FIG. 104 is a flowchart 1,780, illustrating the steps of automatedsample area measurement. In step 1,781, an image of the sample iscaptured. In step 1,782, the maximum number of consecutive pixels thatdisplay a portion of the sample in a first dimension are determined. Instep 1,783, the maximum number of consecutive pixels that display aportion of the sample in a second dimension is determined. In step1,784, the maximum number of consecutive pixels that display a portionof the sample in a first dimension is multiplied by the maximum numberof consecutive pixels that display a portion of the sample in the seconddimension to calculate the approximate area of the sample.

FIG. 105 is a flowchart 1,790 illustrating the steps of automated samplevolume measurement. In step 1,791, a plurality of images of a sample arecaptured. In step 1,792, the width of the sample is determined based atleast in part on one of the plurality of captured images. In step 1,793,the length of the sample is determined based at least in part on one ofthe plurality of captured images. In step 1,794, the height of thesample is determined based at least in part on one of the plurality ofcaptured images. In step 1,795, the width, length, and height of thesample are multiplied together to calculate the approximate volume ofthe sample.

FIG. 106 is a flowchart 1,800 illustrating the steps of automated sampleweight measurement. In step 1,801, a plurality of images of a sample arecaptured. In step 1,802, the width of the sample is determined based atleast in part on one of the plurality of captured images. In step 1,803,the length of the sample is determined based at least in part on one ofthe plurality of captured images. In step 1,804, the height of thesample is determined based at least in part on one of the plurality ofcaptured images. In step 1,805, the width, length, and height of thesample are multiplied together to calculate the approximate volume ofthe sample. In step 1,806, the approximate volume of the sample ismultiplied by an estimated weight per unit volume of the sample, therebycalculating an estimated weight of the sample.

In another embodiment, where a third dimension, such as height, is notmeasured, the weight can be measured by multiplying the approximate areaof the sample by an estimated weight per unit area of the sample. Inthis fashion, the measured area of the sample can be used to calculatean approximate weight of the sample. While calculating approximateweight based on area, and not the volume, of the sample is lessaccurate, in many situations this level of estimation is sufficient toyield useful data.

The estimated weight per unit area may be generated by measuring thearea of multiple control samples for which the weight is already knownand then processing the average or median ratio of the measured area andknown weight data pairs to generate an average or median weight per unitarea based on the control samples.

The estimated weight per unit volume may be generated by measuring thevolume of multiple control samples for which the weight is already knownand then processing the average or median ratio of the measured volumeand known weight data pairs to generate an average or median weight perunit volume based on the control samples.

The above methods of measuring area, volume and weight of samples usingan optical inspector provide valuable information. The above methods maybe performed using an in-flight 3D inspector or an adaptable inspectionunit as disclosed herein. Moreover, these methods of measurement may beused by other optical inspectors as well.

Although certain specific embodiments are described above forinstructional purposes, the teachings of this patent document havegeneral applicability and are not limited to the specific embodimentsdescribed above. Accordingly, various modifications, adaptations, andcombinations of various features of the described embodiments can bepracticed without departing from the scope of the invention as set forthin the claims.

What is claimed is:
 1. A method, comprising: (a) attaching a sortingdevice, a data port, a power port, and an attachment mechanism togetherto form an adaptable sorter unit; (b) affixing the adaptable sorter unitto a first position such that the adaptable sorter unit is capable ofdeflecting a sample traveling along a processing line; and (c) sortingthe sample traveling along the processing line, wherein the sorting of(c) is performed, at least in part, by the sorting device.
 2. The methodof claim 1, wherein the first position is close enough to the processingline such that the sorter device is able to deflect the sample travelingalong the processing line.
 3. The adaptable sorter unit of claim 1,wherein the sorting device includes a vacuum system.
 4. The adaptablesorter unit of claim 1, wherein the sorting device includes a mechanicalpedal system.
 5. The adaptable sorter unit of claim 1, wherein thesorting device includes an air jet system.
 6. The method of claim 1,wherein the attachment mechanism includes a mounting bracket, a mountingbracket receptacle, a weldable material, a clamp, an adhesive, a magnet,a latch, a lock, a locating pin, a rail, a slide, a locking pin, a bolt,or a screw.
 7. The method of claim 1, further comprising: (d)controlling the sorting device; and (e) causing information to becommunicated via the data port.
 8. The method of claim 7, wherein thecontrolling of (d) and the causing of (e) is performed, at least inpart, by a processor circuit.
 9. The method of claim 1, wherein theattachment mechanism is capable to affix the adaptable sorter unit tothe first position proximate to the processing line, and wherein theadaptable sorter unit is not attached to the processing line.
 10. Themethod of claim 1, wherein the attachment mechanism is capable to affixthe adaptable sorter unit to the first position proximate to theprocessing line, and wherein the adaptable sorter unit is attached tothe processing line.
 11. The method of claim 1, wherein the adaptablesorter unit is configured to communicate with an inspecting device viathe data port.
 12. The method of claim 1, wherein the data port is awired communication port or a wireless communication port.
 13. Themethod of claim 1, wherein the adaptable sorter unit receives samplecharacterization data, sorter configuration data, or a sorting commandvia the data port.
 14. The method of claim 1, further comprising: (d)selecting a sorting mode of operation based at least in part on acommunication received from an inspection device.
 15. A method,comprising: (a) attaching a sorting device, a data port, a power port,and an attachment mechanism together to form an adaptable sorter unit;(b) affixing the adaptable sorter unit to a first position such that theadaptable sorter unit is capable of deflecting a sample traveling alonga processing path; and (c) sorting the sample traveling along theprocessing path, wherein the sorting of (c) is performed, at least inpart, by the sorting device.
 16. The method of claim 15, wherein theattachment mechanism is capable to affix the adaptable sorter unit tothe first position proximate to the processing path, and wherein theadaptable inspection unit is not attached to a processing line.
 17. Themethod of claim 15, wherein the attachment mechanism is capable to affixthe adaptable sorter unit to the first position proximate to theprocessing path, and wherein the adaptable inspection unit is attachedto a processing line.
 18. The method of claim 15, wherein the sortingdevice includes a vacuum system, a mechanical pedal system, or an airjet system, and wherein the attachment mechanism includes a mountingbracket, a mounting bracket receptacle, a weldable material, a clamp, anadhesive, a magnet, a latch, a lock, a locating pin, a rail, a slide, alocking pin, a bolt, or a screw.
 19. The method of claim 15, wherein thefirst position is close enough to the processing path such that thesorter device is able to deflect the sample traveling along theprocessing path.
 20. The method of claim 15, further comprising: (d)controlling the sorting device; and (e) causing information to becommunicated via the data port, wherein the controlling of (d) and thecausing of (e) is performed by a processor circuit.